Towards a Conceptual Model to Evaluate Usability of Digital Government Services in Malaysia

The Malaysian government is committed to provide comprehensive digital government services and it is reflected in some policies and strategic plans such as 11th Malaysia Plan 2016-2020 (RMKe-11) for digital government transformation. However, though most of the Malaysia government services are online yet they are still inadequate and the majority of users are unhappy with the current services. Usability is a critical aspect in the success of digital government. Thus, this research aims to develop and validate a usability conceptual model of digital government services in Malaysia context to identify key factors that influence the perceived usability that assists to encourage usage and satisfaction of digital government services. This research has applied quantitativedeductive approach and employed PLS-SEM analysis. Empirical results indicate that Effectiveness, Efficiency, Learnability, Satisfaction, Usefulness, and Citizen Centric are key factors of perceived usability of digital government services. The evaluation of the proposed conceptual model yielded that three of the six factors which are Effectiveness, Satisfaction, and Citizen Centric have significant positive influence on perceived usability of digital government in Malaysia context. Keywords—Digital government; citizen-centric; quantitative; usability


I. INTRODUCTION
Technology can be used by the organisation to permit faster response to customer enquiries and problems, to reduce labour costs, to improve internal efficiency and productivity, and to gain distinctive and differentiating competitive advantages [1,2]. Digital government generally refers to the use of information and communication technologies (ICTs) in government to improve service delivery and improve relationships with citizens, civil society, and private sector [3]. Digital government services consists of online services, mobile applications, big data, open data, social media, digital media, and cloud computing [4]. Malaysian e-government services have been evolutionised from e-government 1.0 in 1995 with static government websites for accessing information to egovernment 2.0 in 2007 where online services are provided for relevant public services transaction among the citizens. In 2015, e-government 3.0 also known as digital government was introduced [5] with dynamic service delivery where government online information services are generating opportunities and innovations through the citizens' participation. The new digital government strategies are stated in the Malaysian Public Sector ICT Strategic Plan (PSISP) 2016-2020 with theme "Citizen Centric Digital Services" and vision "Inclusive Digital Government Drives Citizen Centric Service Delivery". However, Digital Government Satisfaction Survey 2014 by The Boston Consulting Group has reported that only 30% of respondents are satisfied with the Malaysian government services that are provided through Internet and 56% of the respondents rated the quality of the government online services are worse than the private sector and only 4% said that the government online services are much better than private sector [6].
Usability is a critical factor in the success of digital government [7]. Usability is one of the challenges in developing digital government services because usability affects citizens' usage and acceptance of the digital government [8] and may influence their electronic interaction with the government. Usability can improve users' trust in the digital government [7] and affect the credibility of the digital government services [9]. Trust in digital government websites is associated with perceived website quality [10] which means the websites are technically reliable and ease to use. The government needs to concern about usability because it will affect the user experience and users' trust in the digital government services [11]. The digital government services through websites represent their physical office of government agencies. High usability of the digital government services shows that the government is committed to deliver their services to fulfil the citizens' needs and demands. There are some studies have been conducted regarding usability factors of the e-government or digital government where the majority of these services are about election and voting website, local government website [7,9,[12][13], e-learning [14], and digital library [15][16]. Besides, to our best knowledge, there are only a few studies in Malaysia context reporting on the usability of www.ijacsa.thesai.org higher education agencies' websites [17], and student information system [18]. Therefore, it becomes significant to explore what other potential usability factors affecting perceived usability of the digital government services in Malaysia. To answer this question, this study has embarked a quantitative case study approach to deduce more relevant factors of interest and validate the factors.
The remainder of our paper proceeds as follows. The next section we present the reviews of literature on digital government and usability theories, the design of the conceptual model and the development of the hypotheses. Section III presents the methodology detailing our data collection and analysis procedure. We then discuss our findings in Section IV and this paper end with brief concluding remarks, limitations, and recommendations for future research in Section V.

A. Digital Government in Malaysia
Digital government is a way of digitalizing government services to improve the service delivery system in an efficient and effective way to maintain a relationship with the citizens. Digital government leads to improvement of government function, services, and works to provide comprehensive service delivery that will satisfy the citizens. The digital government enables low cost of communication between government and citizens through digital platform such as websites, online services, social media, and mobile application. This effort was expected to improve citizens' views on governments [19], especially in term of efficiency and trust. The government has to concern about trends and rapid changes of technology to ensure the digital government services are meeting citizens's demands. The government faces a challenge to grasp the speed and scope of changes in technologies like some emerging technologies such as the Internet of Things, data analytics, and artificial intelligence which have a great potential for government to improve their service delivery [20]. Besides that, digital communication should offer a faster and efficient response from the government [21] but the government needs to put adequate concern on technological issues such as interoperability, flawed configurations, and misalignment with work processes.
Recently, the Malaysian government has announced a nationwide development plan known as "RMKe-11" for five years' period from 2016 to 2020. The RMKe-11 focuses on rapidly delivering high impact outcomes to both the capital economy and people economy at an affordable cost. RMKe-11 [22], is aimed at transforming public service for productivity through and enhanced ICT and digital government.
In proposing the usability conceptual model of digital government services in Malaysia, we first reviewed usability baseline theories and models to identify usability factors from existing theories and models, coupled with a review of literatures especially in digital government context. Finally, we proposed a conceptual model which are to be validated subsequently. Fig. 1 illustrates our literature review process. We described the overall process and results of literature analysis in the following sections.

B. Usability and Related Theories
Usability is one of the critical factors that influence successful system development and implementation. In this study, we define usability as efficiency and effectiveness of a system that meets user needs and expectations, and the system is able to satisfy the user to perform and complete their task.
There are several related theories and models were introduced regarding the usability. Shackel [23] proposed a usability model by integrating the effectiveness of system, learnability, flexibility, and attitude of system users. ISO 9241-11 is standard used to measure usability in term of effectiveness, efficiency, and satisfaction [24]. Indeed, ISO 9241-11 Standard is one of the usability frameworks that were cited widely in usability engineering field. ISO/IEC 25010:2011 [25] is another usability measurement which is a part of System and Software Quality Requirements and Evaluation (SQuaRE) series of International Standards. There are five quality characteristics defined by ISO/IEC 25010:2011 which are effectiveness, efficiency, satisfaction, freedom from risk, and context coverage. Nielsen [26] has developed a model of the attributes of system acceptability and listed five factors that affect usability which is easy to learn or learnability, efficient to use, easy to remember, few errors, and subjectively pleasing and Nielsen's Usability Heuristics [27]. Quesenbery [28] defined usability as "quality or characteristic of a usable product". The author introduced the dimension of usability that consists of 5Es which are effective, efficient, engaging, error tolerant, and easy to learn. Recently, Rubin and Chisnell [29] identified six factors such as usefulness, efficiency, effectiveness, learnability, satisfaction, and accessibility that affects usability.

C. Related Works
The existing usability theories and models mentioned in the previous section (section B) have been considered and applied appropriately by several researchers for usability evaluation and measurement in their study. For example one study Brown et al. [30] proposed a usability evaluation model to evaluate mobile health technology. There are eight concepts included in their model and these are error prevention, information needs, memorability, learnability, competency, performance speed, flexibility/customization, and other outcomes. Another study [31] investigated the usability on social media sites' when adopted for business-to-business (B2B) marketing purposes in China. The authors' investigation found that three factors such as learnability, efficiency, and memorability impact the perceived usability. Usability framework was proposed by www.ijacsa.thesai.org another study to evaluate the usability of a smartphone application to indicate prevention and early intervention of anxiety in youth. This evaluation used five dimensions of usability, which are ease of use, ease of learning, quality of support information, satisfaction, and stigma [32]. Other studies conducted usability testing through explorative, descriptive design on Facebook-based obesity prevention program for Korean American adolescents (Health Teens). The usability testing involved one-on-one observation, the thinkaloud method, audiotaping, screen activity capture, and survey. The authors [33] applied Technology Acceptance Model (TAM) for usability evaluation of the program. Some authors also applied or adapted the existing usability theories and models mentioned earlier in the e-government or digital government context. A study evaluated 120 samples of higher education institutions' websites through the Malaysian Ministry of Higher Education's portal. The usability evaluation was conducted on some criteria and these are page size, speed, and broken links [17]. Meanwhile, another study proposed a model of usability measurement for an academic digital library. The authors adopted three major factors from ISO 9241-11 Standard namely efficiency, effectiveness, and satisfaction and one major factor from Nielsen's Model of the Attributes of System Acceptability namely learnability.
Additionally, student information systems usability was analyzed by [18], and factors that affect the system usability are useful information, timely access, interface design, and error recovery. In [14, author used Shackel's Usability Model comprising four usability factors including effectiveness, learnability, flexibility, and attitude to assess the usability of elearning system. Furthermore, [9] evaluated three local egovernment websites using criteria from Nielsen's Usability Heuristic and added three more criteria which are interoperability, support and develop users and pleasurable and respectful interaction with users. [16] used six factors of usability which are usefulness, efficiency, effectiveness, learnability, satisfaction, and accessibility to evaluate the usability of the Central Science Library (CSL) website. Meanwhile, [12] build 15 web usability standards with six general areas which are overall design standards, hypertext, navigational standards, readability, language option, and findability in evaluated 34 websites from the list of Alabama counties which are specifically for voting and elections. Also, [13] used five usability categories which are overall design standards, conventions for hyperlinked text in the main text, navigational standards, findability, and readability. Meanwhile, [34] conducted a study that examined the usability of six different electronic nursing record systems. The usability testing was conducted based on five measurements which are efficiency (relevancy), proficiency (accuracy), competency index, additional entry, and time consumed. [35] used usability guides were derived from ISO 9241-11 Standard namely effectiveness, efficiency, and satisfaction to measure usability of open course ware (OCW) initiative from Universidad Tecnica Particular de Loja, Ecuador.
From the above discussion, there are some common usability factors appeared in various theories and models which also appeared as the measurement in the evaluation of egovernment or digital government usability by some researchers. Thereby, after reviewing baseline theories and related works we identified five factors (effectiveness, efficiency, learnability, satisfaction, and usefulness) that may affect the perceived usability of digital government services in Malaysia. Table I shows the frequency analysis of common usability factors found in prior studies.  [24,25]. Effectiveness focuses on process interaction viewpoint with regard to system accuracy and completeness that leads the user to achieve specified goals. Effectiveness is an important factor when providing digital government services as effectiveness can measure how well digital government services meet user expectations.
2) Efficiency: Efficiency refers to the resources expended about the quickness, accuracy, and completeness. Efficiency is considered an important factor because efficiency measures the ability of the digital government services to assist the user in performing their task.
3) Learnability: Learnability refers to ease of teach and learn where the novice users should be able to use the system after a specific time or specified training, and retention of skills for the casual users. Learnability is considered an important factor as easy to learn of the digital government services may encourage the user to return using digital government services. 4) Satisfaction: Satisfaction focuses on positive attitude which includes comfort and acceptability from the user viewpoint. In other words, users are subjectively satisfied when using the system or services. Satisfaction is considered an important factor as satisfaction has the potential to encourage user loyalty which also will increase the usage of digital government services. 5) Usefulness: According to usefulness is "the degree to which a product enables a user to achieve his or her goals, and is an assessment of the user's willingness to use the product at all." Usefulness refers to the quality of the information, easily to understand the information, and capability of the information to help the user complete their task. Usefulness is considered an important factor because usefulness may lead to easy of understanding the information in the digital government services which also will assist the user in performing their task.
6) Citizen centric: In addition to the five factors identified, we also added Citizens-centric as one of the factors www.ijacsa.thesai.org to be included in the proposed usability conceptual model of digital government in Malaysia environment. The digital government services have transformed from governmentcentric approach to citizen-centric approach which widely applied in many countries such as United States of America, Canada, England, Australia, Italy, South Korea, and Singapore [36]. Furthermore, [37] stated that citizen-centric evaluation is necessary to improve the usability of digital government services where the government needs to reflect the feedback or opinion from the citizens in redesign or re-engineering of digital government services. In regard to Malaysia context, [38] have proposed citizen empowerment as one of the success factor in digital government implementation. Hence, the citizen-centric approach is essential in the development of sustainable digital government to improve the service delivery. The importance of a citizen-centric approach in digital government services are as follows [39,40]:  To deliver services that meet citizens needs, who are the primary user.
 To focus on design and development of effective digital government services to meet the citizens needs.
 To diversify channels to provide a choice and convenience for citizens to access government services.
 To take into account the provisioning of government information to meet the needs of different levels of stakeholders.
 To encourage citizens's e-participation and response.
Based on previous studies related to usability in related domain, many researcher have highlighted the importance of usability features and factors from common usability literature and guidelines. This study found that there is a significant need to consider citizen-centric factor in the proposed usability model.  We then acknowledged that there is a need to conduct a study related to the usability of digital government services in Malaysia. Therefore, we continued the research by readings on various academic journals using Science Direct, Web of Science, IEEE Xplorer Library, Google Scholar, and Google. We then applied a deductive approach based on quantitative research design. Deductive approach is concerned with the reconstruction of existing theory to create a new one [41]. The deductive approach involved exploring a known theory or phenomenon and tests if the theory is valid in a given circumstance. There are some stages in deductive approach, which are theory review, deducing hypothesis from theory and formulating hypothesis, testing the hypothesis using relevant method, and examining the outcome of the test to confirm or reject the hypothesis [42].

A. Case Selection
Our research focused on digital government services provided through MyGovernment Portal (www.malaysia. gov.my) such as tax and compound payment, business registration, membership registration and renewal, and application of scholarship in Malaysia. We selected MyGovernment because of this portal acts as a single gateway of Malaysian digital government for information, transaction, and communication with the government.

B. Research Instrument Design
Instrument development for survey data collection of this research was based on Google Forms. The instrument design was based on factors shown in Fig. 2 which consists of six independent variables namely Effectiveness (ET), Efficiency (EC), Learnability (L), Satisfaction (S), Usefulness (U), and Citizen Centric (CC), and a dependent variable, namely, www.ijacsa.thesai.org Perceived Usability (PU). The items used in the instrument were designed and adopted from previous digital government studies.
In prior, three experts were involved to review the content of the survey and the analysis was performed using content validity index (CVI). The initial survey instrument consists of two sections with a total of 39 items or questions. Section A collects Demographic Information, consists of eight items using Scale and Category. Meanwhile, Section B represents Usability measures comprises 31 questions and were reviewed in prior by three experts to evaluate the relevancy and reliability of the items by using CVI approach. The expert review analysis applied rating that advocated by [43], which are scaled as Not Relevant (1), Somewhat Relevant (2), Quite Relevant (3), and Highly Relevant (4). A 4-point scale is suggested to avoid having a neutral and ambivalent midpoint [44]. As a result, all items in this study have achieved the CVI of 3 or 4 by the experts and met the CVI criteria as suggested by [45] and [46]. Mean of Item CVI is 0.89 and it is the same value for Mean of Expert Proportion. Subsequently, the actual survey instrument was also improvised based on the comments and suggestions from the experts with a total of 37 items or questions. The actual survey has been revised and improved containing 37 questionnaire items. In particular, Section A consists of 8 items, and Section B consists of 29 questions (see Appendix A for detail).

C. Data Collection and Analysis
This research applied a non-probability sampling namely purposive sampling. Citizens of Malaysia are the target population engaging individual perspective on perceived usability of the Malaysian digital government. Descriptive analysis was adopted in analyzing the quantitative data in the survey. Descriptive analysis answering questions are about who, what, where, when, and to what extent. Descriptive analysis helps to view data in correct context, identify relevant information in the data, assess the quality of the data, and recognize the assumptions, limitations, and generalizability of the findings [47].
We applied PLS-SEM as a statistical tool that supports multivariate analysis and able to simultaneously analyse multiple variables. By following the guidelines by [48], we chose PLS-SEM for four reasons: Firstly, development, exploration, and testing of usability conceptual model of digital government in Malaysia environment which consists of six exogenous constructs which are Effectiveness, Efficiency, Learnability, Satisfaction, Usefulness, and Citizen Centric, and one endogenous construct which is Perceived Usability. Secondly, no identification issues with small sample sizes where the minimum size is 30. Generally, this technique can achieve high levels of statistical power with small sample size. Thirdly, this technique is highly robust as long as the missing value is below a reasonable level. The missing value is impossible because all items and questions in the online survey are set as mandatory. Fourthly, this technique can easily incorporate reflective and formative measurement model to assess the relationship between constructs and their indicators.

A. Demographic Analysis
Demographic profile for this research consists of eight variables and these are gender, age, educational level, field of study, Internet experience, Internet usage per day, digital government experience, and digital government usage. From the survey, 22 (33.8%) of the respondents are male, and 43 (66.2%) respondents are female. The big percentage of respondents' age ranges between 31 to 40 years old which is a total of 52 (80.0%) respondents. In addition, 31 (47.7%) respondents have Bachelor Degree as a highest academic level, while 20 (30.8%) respondents are Master Degree holder. 46 (70.8%) respondents are from IT field, and seven (10.8%) respondents are from the business field. The survey also shows that 60 (92.3%) respondents have experience of using the Internet for more than five years and spending more than three hours using Internet per day. Furthermore, 41 (63.1%) respondents have experience of using digital government for more than five years, and 31 (47.7%) respondents are using digital government daily or almost daily.
PLS-SEM involves two stages of assessment which are the assessment of the measurement model and assessment of structural model. Following sections of this paper describes these two stages in details.

B. Measurment Model
Measurement model describes the relationship between constructs and their corresponding indicator variables. The measurement model of this has six exogenous constructs which are Effectiveness (ET), Efficiency (EC), Learnability (L), Satisfaction (S), Usefulness (U), and Citizen Centric (CC) and one endogenous construct which is Perceived Usability (PU). Each of the variables is measured by multiple indicators using Likert Scale from 1 to 5 (Strongly Disagree to Strongly Agree). In particular, the type of this measurement model is reflective which all the indicators are caused by the same construct [48]. Assessment of reflective measurement model includes composite reliability to evaluate internal consistency, individual indicators reliability and average variance extracted (AVE) to evaluate convergent validity, and Fornell-Larcker criterion and cross loadings to assess discriminant validity.
Internal consistency reliability refers to composite reliability values in which higher value indicates higher level of reliability. All constructs in the measurement model have composite reliability value higher than 0.708 with the highest is Usefulness (0.963). Furthermore, the individual indicators reliability is examined through outer loadings relevance testing. A higher value of outer loadings indicates that the construct has much in common [49]. From the PLS calculation, outer loadings value for all indicators are higher than 0.708 except for B-S3 (0.689) and B-PU1 (0.705).
AVE is used to measure convergent validity on the construct level. AVE value of 0.50 or higher indicates that, on averages, the construct explains more than half of the variance of its indicators. Meanwhile, AVE value below 0.40 indicates www.ijacsa.thesai.org that, on average, more error remains in the items than the variance explained by the construct [48]. From the result, all constructs have AVE value higher than 0.50 with the highest is Usefulness (0.839). Fornell-Larcker criterion and crossloadings check are used to measure discriminant validity. For Fornell-Larcker criterion, the square root of the AVE of each of the construct should be higher than its correlation with any other construct.
Our results show that all model evaluation criteria have been met except for Efficiency → Perceived Usability, and Satisfaction→ Perceived Usability based on Fornell-Larcker criterion. Furthermore, cross-loadings results show that an item's loadings on its own construct in all cases are higher than all of its cross loadings with other constructs. Thus, the discriminant validity between all the constructs is based on the cross loadings criterion (see Appendix A for detail). Besides that, outer loadings of B-S3 and B-PU1 are lower than the suggested value. However, the Efficiency and Satisfaction constructs, and the B-S3 and B-PU1 indicators are still retained because the respective composite reliability and AVE are higher than suggested value. Table II shows the summary of measurement model results.

C. Structural Model
Structural model assessment involves examining the model's predictive capabilities and the relationship between the constructs. We followed rules of thumb for structural model evaluation recommended by [48]. We first assessed structural model for collinearity issues. We then examined the significance of path coefficients to understand the relevance of the structural model relationships followed by measuring R² to see predictability of the overall model. In addition, we also assessed the effect size f² which allows examining an exogenous construct's contribution to an endogenous latent variable's R² value. Finally, we measured predictive relevance (q²) of the endogenous construct.
Our collinearity assessment is measured based on VIF (variance inflation factor) value where the value should be between 0.20 and 5.00. Table III shows the VIF value for all six constructs. Five of the constructs are between the suggested value, while Efficiency is 5.148 which is higher than 5.00. Hence, the Efficiency is eliminated for the next assessment.
Path coefficients represent the hypothesized relationships among the constructs. The bootstrapping is used to assess the significance of path coefficients where the bootstrap samples are 5,000 [49]. In addition, the critical value (t) applied for this research is 1.96, p<0.05 (significance level=5%). Our results show that Effectiveness, Satisfaction, and Citizen Centric are significant whereby Learnability and Usefulness are not significant (p>0.05). Table IV shows the significance testing results of the structural model path coefficients.
Furthermore, the R² value is 0.828 which is higher than 0.75 that indicates as substantial. The f² effect size of Citizen Centric is higher (Effectiveness: 0.142, Satisfaction: 0.323, Citizen Centric: 0.376) than the recommended threshold 0.35 which indicates that the exogenous construct has a large effect on the endogenous construct namely Perceived Usability. The q² effect size is 0.912 which is higher than 0.35 that indicates the exogenous constructs have a large predictive relevance for the endogenous construct (Perceived Usability). Digital government was introduced in Malaysia in 2015 with dynamic service delivery where government online information services are generating opportunities and innovations through citizens' participation. Perceived usability should be emphasized to ensure the success of digital government development and meet the citizens' needs and expectations. The essential contribution of this research is the integration of existing key elements of usability theories and models, and digital government services to propose usability conceptual model of digital government in Malaysia environment. The second contribution is the validation of usability conceptual model of digital government in Malaysia environment. This research has validated that three of the six factors which are Effectiveness, Satisfaction, and Citizen Centric are relevant and significant on the perceived usability of digital government in Malaysia. Meanwhile, Efficiency is eliminated during the assessment to treat collinearity problems, and Learnability and Usefulness are indicated as not significant. Development of this conceptual model is started by conducting the literature review and preliminary study. Then, expert review was conducted for the content validity of the survey instrument. Subsequently, data collection through was conducted through an online survey among citizens as target respondents. Finally, the conceptual model was evaluated and validated using PLS-SEM technique. The conceptual model has three factors that are significant on the perceived usability of digital government in Malaysia environment. The model could become guidance for the Malaysian government to revise and develop a strategy for the sustainable digital government to improve public sector service delivery system and align towards the high-income nation. Although the model is developed in the Malaysian government context, the model may be useful also for industries who involves in the development of digital services.
However, there are some limitations to this research. Firstly, survey instrument design of this research only involved content validation through expert review. Secondly, the data collection process which was conducted in one time due to time and cost constraints. The more accurate data will be collected if data collection is conducted more than one time. Another limitation of this research is regarding the purposive sampling method as the survey questionnaire cannot be spread widely and in a longer time to get more variety of respondents' background. Finally, this research is conducted on citizens perspective on perceived usability of digital government in Malaysia environment which more on non-functionality of system usability. Therefore, this model does not cover much on the functionality of system usability such as interoperability, accessibility, navigational, error prevention/recovery, and speed. The digital government services are well designed.

B-EC3
The digital government services can save citizens's time.

B-EC4
The digital government services can save citizens's expense.
Learnability B-L1 I am quickly becoming good at using the digital government services.
[32], [52] B-L2 I learn to use the digital government services quickly. B-L3 I can easily remember how to use the digital government services. B-L4 The information on the digital government services is clear.

B-L5
The information on the digital government services is easy to understand. B-L6 In general, it is easy to learn to use the digital government services.
Satisfaction B-S1 The digital government services are high quality in term of accuracy or trustworthy.
[32], [53], [54] B-S2 I am happy with the digital government services. B-S3 I plan to use the digital government services in the future. B-S4 I am satisfied with the way that the digital government services have carried out transactions.

B-U1
The digital government services are more convenient than traditional system. [55], [56] B-U2 Using the digital government services enable me to accomplish the required task more quickly. B-U3 The digital government services allow me to accomplish more transactions with the government. B-U4 The digital government services increase my productivity and efficiency. B-U5 In general, the digital government services are useful for me.