A Study on Usability Awareness in Local IT Industry

Usability awareness receives more consideration by industry professionals and researchers throughout the world, but it is limited in Pakistan. This study reports survey results of the current state of usability awareness in the local Information Technology (IT) industry. Forty participants – IT practitioners from IT industry – were involved in the study. We used Usability Maturity Model (UMM) and content analysis methodology to discover the current status of usability awareness. The results indicate that 1) almost half (18 out of 40) of the participants were unaware of the term usability and related concepts, 2) there is shortage of HCI/Usability professionals in organizations, 3) most of the software companies were at unrecognized level of UMM and 4) they were also not interested in usability because of limited or no budget for it. The study also reveals a gap between usability awareness and its perceived usefulness among IT professionals. Keywords—Usability; usability awareness; human-computer interaction (HCI); HCI practitioners; Pakistan IT industry


INTRODUCTION
Human Computer Interaction (HCI) deals with the knowledge to assist human's physical and mental skills for the ever-developing technology [1].In Europe and United States, HCI is playing a vital role in IT industry since 1980 [2].One major topic of HCI is the study and practice of usability.Usability is a quality characteristic that includes many factors e.g. a product should be easy to learn and use [3].The scope of usability is not limited to user interface only rather it deals with the entire system [4].
A lot of work related to usability has been done e.g. the study [5] provides an overview of different usability evaluation methods used for web applications.Another study [6] also presents the various user experience (which includes such as usability) evaluation technologies for software applications.Similarly, in a recent systematic literature review [7], reporting mechanisms for usability defects are summarized and discussed.
Despite of all the work done, awareness about the usability methods and practices is not good and the role of HCI practitioners is overlooked [8].According to [9], usability awareness means "designing for a sustainable world".In general, awareness guides towards the maturity, eight levels of usability maturity (like software process maturity) for an organization are proposed in [10].If the system is developed by the organization having the high level of usability maturity, the end user will use the system without any training required for it [11].
There is continuous appreciation and acceptance for usability in organizations.Many studies e.g.[11]- [14] have been conducted to assess the usability maturity of industry in different countries.These studies are mostly conducted in developed countries like Germany, Japan, and Israel.There is a need to conduct similar studies in other countries particularly the developing ones to assess the usability awareness and maturity in general.This study aims to explore the current state of usability awareness in the local IT industry of Pakistan.The findings of this study may guide the academia and other concerned authorities of the developing countries (Pakistan in particular) for better planning to cater the usability needs of the industry.

II. RELATED WORK
Many studies have been conducted about the usability awareness and practices in different parts of the world.A survey was conducted in Malaysia about the usability awareness of IT and non-IT practitioners [8].Out of total 72 participants, 23 IT practitioners, 27 IT scholars and 22 non-IT experts participated in this study.The results revealed no major differences of usability awareness among IT practitioners, IT Scholars and non-IT professionals.Study participants also considered usability as God-gifted skill and common-sense knowledge for both IT and Non-IT staff.Another survey was conducted in UAE where the participants were IT managers, marketing professionals and end users [11].The results revealed that the participants had introductory www.ijacsa.thesai.orgknowledge about usability and they did not study it as a significant role-player for software development.Furthermore, no user involvement in design phase and unavailability of usability staff were reported.
A total of 72 participants were involved in a survey conducted in Korea [15].The participants were software developers, usability and user interface professionals.The results showed that usability had not been applied in projects, but the increased awareness of usability was stated.The lack of usability professionals, time and cost-effective usability methods were also reported as the main problems.A study was conducted in France to find the perspective of HCI professionals about usability methods [16].The results revealed that professionals were aware of usability methods after many years of working in industry.Results also showed that young experts didn't use usability methods and approaches in developing user interfaces at their earlier stages of profession.The specialists from engineering schools were more aware of usability as compared to other graduates.
The primary aim of the study was to find the practices, awareness level and perceptions of User Experience Professionals (UXPs) about web accessibility in Turkey [17].An online survey study was performed to meet the primary goal of the study.The finding indicates that UXPs have confidence in that they have enough education and training regarding web accessibility.But, they were not aware with web accessibility standards and were not considering to apply them in their projects.They think that considering the web accessibility is the responsibility of the project managers.The study conducted in German institutes highlighted the current practices, usability awareness, perceptions of usability and networking strategies in Germany [18].The institutions showed the openness to accept recommendations and suggestions to optimize the interface designs/usability in general.The shortage of budget was considered the main problem to carry out the usability related tasks.
The aim of the study was to find the current practices in the Nigerian software industry [19].A survey study and semistructured interviews were performed in local software houses.The results show intermediate level of usability awareness and the limited knowledge of HCI practices.Another study was conducted to find the understanding and awareness level of HCI [20].The results show that HCI practices and processes are at their beginning level in most of software houses.The results also indicate that, during software development, the end user involvement was also not considered.Furthermore, it was found that there was lack of HCI knowledge transfer to university students.
In a survey conducted in Brazil, the attempt was to find out the opportunities and challenges in HCI education [21].109 participants contributed in the survey.One of the biggest challenges was to get a good HCI position in industry after completing the degree.The findings also highlighted the importance of continuously updated knowledge about the latest technology development and active collaboration among the faculty members.Another study in New Zealand indicated the absence of proper schooling, information and skills about usability approaches, procedures and practices between designers and developers [22].According to a study, usability community in Russia had been facing many problems e.g.lack of professional training and standards and insufficient awareness among professionals [23].

III. METHODOLOGY
A survey was conducted to assess the level of usability awareness and maturity in the local IT industry.For this purpose, a questionnaire was designed based on the Usability Maturity Model (UMM) [24] which helped to assess usability as the ability of an organization.UMM defines six levels of usability maturity i.e.Level X (unrecognized), A (recognized), B (considered), C (implemented), D (integrated), E (institutionalized). Level X is the lowest level which indicates the organization's negligence towards usability whereas Level E is the highest level of usability maturity within the organization.The following questions were asked from participants to find out the current state of usability awareness and the results were mapped on UMM maturity levels.
~ Is there any HCI/Usability staff in your organization?~ Users are involved in design phase of the system?~ Is there any budget allocated for usability related activities in your organization?~ Does your organization consistently produce usable products?
~ Does the top management of your organization focus on design for human use?

A. Participants
The questionnaire was sent through email to professionals of 65 organizations located in different cities of Pakistan.In total, 40 participants responded with their feedback.The detail of their professional roles is given in Table I.Most of the participants (about 82%) were less than 30 years of age.The remaining (about 18%) participants were between 31-40 years of age.In terms of working experience, 2 participants (5%) had 10 years of job experience; 13 participants (32.5%) with 5 years; 9 participants (22.5%) had an experience of 3 years; 7 participants (17.5%) with 2 years of experience; and 9 participants (22.5%) had 1 year of job experience.The data we received as a result of survey is mostly qualitative and thus sits valid for content analysis [25].The specific context of this data describes the IT industry's overall perception and awareness regarding usability.

A. Usability Awareness in General
Based on the separate question; "Have you ever heard of usability?"18 participants out of 40 (45%) were not very familiar about the usability.Interestingly, most of them were computer programmers.The possible reason is their inclination towards coding rather than designing user interfaces which are not given the due priority (by the programmers themselves and also by the software development organizations).The remaining 22 participants (55%) were well-aware about the usability for product design.

B. Organizations' Capability to Handle usability Issues
The results of this study show that many organizations have no staff for handling issues in usability or in user interface design.In such organizations, software developers perform all tasks.Usability issues are not properly handled due to lack of usability practitioners.Furthermore, there seems to be no formal usability training, for software developers, to handle usability related issues.

C. Development of Usable Products and Budget for Usability
The development of usable product is directly related with the budget allocation for this purpose.In absence of budget allocation, it is difficult to hire usability experts or performing other usability related activities.More than 50% (23 out of 40) participants indicated that their organizations had no budget for handling usability related activities.

D. Usability Experts Inside Organization
It is important to know the availability of HCI/usability experts in an organization which indicates the seriousness of that organization towards usability.15 participants (37.5%) informed about the unavailability of usability professionals in their organizations.It reveals that usability is not considered an important aspect in those organizations.The remaining 25 participants informed that their organizations have HCI professionals.These HCI professionals were hired for different positions and levels.9 participants (out of 40, 22.5%) informed about the availability of user experience professional in their organizations.Five participants (12.5%) shared that use the titles of interaction designer and usability expert in their organizations.Two participants informed that the titles of usability engineer and HCI expert were used in their setups.

E. End user Involvement in System Design and Top Management Commitment Towards Usability
User participation and approval is valuable for system success [26].There is a low chance for a system to be userfriendly if users are not involved in the designing phase of the system.An active and frequent user participation throughout the system development is a basic principle for User-Centered System Design (UCSD) [29].13 participants (32.5%) mentioned that their organizations did not involve users in system design.The rest of participants (67.5%) claimed that their organizations focused on users' participation during the design process.
V. UMM MAPPING Each question, described above, carried a weight equal to 1.The participants answers (Yes or No; and/or the indication of HCI/Usability staff in either Yes or No) were assigned equal weights and then summed up to maximum value of 5.The companies which achieved the score of 5 have the highest level of usability maturity.Fig. 1 describes the percentage of IT companies and their achieved level of UMM maturity.It explains that 31% of the companies achieved the average level of UMM maturity while there were 6% of the companies who were at unrecognized level (X).If we consider level C as the threshold then 57% of the companies were below that threshold (by counting the results of level X, A, B and C respectively).

VI. DATA ANALYSIS
This analysis provided main themes, categories and general insight on the usability awareness of the local IT industry.We performed the analysis on one part of survey data i.e. for the question; your perception of the concept of the usability.The scope of the question is limited to the data received from the professionals (population of this study) of local IT industry.We focused on these professionals because they work in industry where there is a need of applying usability and its principles.The other factors that describe the population include age, gender, profession, education and domain experience.
For the data (i.e.responses), received from participants, we used emergent and a priori coding techniques [27] to analyze it and found out interesting themes.In emergent coding we www.ijacsa.thesai.orgperformed the analysis of the responses and extracted the themes without considering existing HCI/Usability theories or models.It means, the resultant themes were actually the subjective list of participants-defined words.In a priori coding we performed the analysis of the responses and extracted the themes based on the usability and Human Computer Interaction concepts commonly found in the relevant literature.Two coders coded the responses independently in order to avoid the bias in the analysis.The result of this analysis is described in the Table II.
The data was then further cleaned and few entries were removed from responses.This was based on the participants' answers which were not very clear.For example, few responses were like this; "I don't have any idea", "Good", "don't know", etc.These responses were discarded because otherwise they would have created the anomalies in the final result.Thus, the total of 27 valid entries (Table II) were considered for further analysis.Both coders, independently, coded all 27 entries and found out similarity in 12 emergent themes and in 9 a priori themes (highlighted in Table II).While, only 7 entries were coded similarly in both types of coding.

A priori coding
Emergent coding

A priori coding
3 importance usability usability is important for easy use ease of use 4 user-friendly, ease of use user-friendly ease of use by focusing users' real tasks ease of use 6 quality improvement usability improved quality product and user's comfort user satisfaction 7 ease of use ease of use, understandab ility easy to understand and use ease of use 8 ease of use ease of use, understandab ility easy to understand and use ease of use As the results from this qualitative data analysis and its interpretation could involve researchers (coders) subjectivity, thus we need to be sure to describe the data reliability.This was achieved by performing inter-coders reliability [25] in terms of % of agreement and disagreement as: % agreement= the number of themes coded the same way by different coders/the total number of themes % agreement (emergent) = 12/27 = 44% % agreement (a priori) = 9/27 = 33% The results indicate that both coders agreed on less than half of the themes.This further strengthens our argument of performing the coding independently and thus obtaining the results without bias.Furthermore, it describes that the identification of themes is probably dependent on the number and richness of responses along with the coders and participants knowledge in the same domain.www.ijacsa.thesai.org In order to converge our results to few categories we then compared the themes and extracted the occurrences of similar themes.This also helped in identifying and defining the relationships between themes, and thus creating a set of categories which is called the code list (or nomenclature) [25].This code list provides a hierarchical structure of the themes with multiple levels of details.In the word cloud (Fig. 2) and code list (Fig. 3, [28]), the unique occurrences i.e. the most prominent words in word cloud are; usability, ease of use, understandability, and quality improvement.Some themes are reported more than once and do not add up to the total number described in the parent node of the code list.This word cloud and the code list helped us in identifying the usability-relevant terms and describe the overall attitude, trend and inclination of IT industry towards usability awareness in-situ.In Fig. 3, code list created using emergent and a priori coding, describes the concepts currently exist in local IT industry.The numbers in parentheses represent the number of occurrences counted in participants responses'.
We also calculated top-2 box score [30] for the question; "How useful do you think is the usability for you (and your organization)?"This question asked after the survey and was measured on 5-point rating scale value (5-very useful ………1not at all useful).Top-2 box score describes the participants who strongly and somewhat strongly agree to the statement/question being asked.The results for this question indicates that almost 47% of the participants considered applying usability is a useful measure, though, 55% of the participants were well aware of usability.This result when combined with the result of the question; "Have you ever heard of usability?"helps in comparing awareness and usefulness.This returns an encouraging result i.e. there exist a smaller usability awareness vs usefulness gap (Fig. 4) which could be filled by following the suggestions/guidelines presented in conclusion.VII.LIMITATIONS OF THE STUDY Some themes in emergent and a priori coding columns (for both coders) look closely related but we didn't count in similarity because both themes represent different levels of detail.For example, for participant 6, one theme is usability www.ijacsa.thesai.org(which is a general concept) while the other is user satisfaction (which is a specific concept within usability).The data analysis, performed by two coders, does not generate an acceptable level of agreement (this is <40% in average for both types of coding) which indicates the need of involving more coders in the analysis process and thus achieving a better reliability score.Furthermore, the word cloud and code list provide the relevant themes currently exist in IT industry, but they do not truly reflect an individual's or organization's intention or struggle towards achieving the goal (i.e.applying usability at their work place).Exploring this struggle and efforts -currently going on in the industryis an important future research direction.

VIII. CONCLUSION
Usability awareness is currently limited in the local IT industry of Pakistan.Before performing this survey study, we assumed that IT professionals automatically gain usability perception and awareness during their work -because they have to deal with the evaluation of interactive systems -but the assumption was not true.It has been found that it depends upon their type of work and the organization's interest in applying usability techniques.Furthermore, in most of the organizations the end users were not involved in product user interface design phase.Lack of HCI/Usability professionals is also observed in most of the organizations.Therefore, based on the participants responses' few organizations could be ranked at recognized level of UMM.It was found that most organizations are not interested in usability because they have no budget for it.
We conclude that the usability perception, awareness and its importance can be achieved by 1) conducting training workshops describing the advantages of usability especially on the Return On Investment (ROI) of the company and 2) consulting higher education institutes and asking them to train students accordingly by focusing more on usability and 3) requesting top management to reserve budget for usability tasks.By following these suggestions, we are hopeful, that this will develop a rich usability culture in IT industry of Pakistan.

Fig. 1 .
Fig. 1.The IT companies and their achieved level of UMM maturity.

Fig. 2 .
Fig. 2. Word cloud: the relevant themes popular within local IT industry.

Fig. 3 .
Fig. 3.The concepts (code list) currently exist in local IT industry.

TABLE II .
E XTRACTION OF THEMES FROM P ARTICIPANTS ' RESPONSES