A Systematic Mapping Study of Software Usability Studies

—Among software quality attributes “software usability” is considered as one of the vital factors in software engineering literature. Software usability is the ability for users to generally understand, use, and learn a software with ease. Due to the importance of usability in software quality, a considerable amount of literature is published in the past decade. Few review and survey studies are also published to critically review the existing literature in the domain. However, there is limited research covering systematic mapping study of software usability. Mapping studies help in analyzing the general trends and research productivity in a research area. To fill this gap, this work critically examines the overall research productivity, demographics, trends, and challenges of software usability. The objective is to classify the current contributions and trends in the area of software usability. We retrieved 9,874 research articles from six research databases and 62 works are selected as primary studies using an evidence-based approach. The result of this mapping study shows that software usability is an active research area, with a promising number of works published in the last decade (2011 - 2020). We identified that the current literature spans over multiple article classes of which investigative papers, model proposals and evaluation papers are the most frequently published article types. We found experiments and theoretical validations to be the most common validation techniques. In terms of application domains; web, software development and mobile applications are the most frequent domains where usability studies are conducted. We identified that future usability studies should focus more on field studies as well as on the usability testing of scientific software packages. It will be of importance to consider ethical issues in usability testing as well.


I. INTRODUCTION
Based on IEEE Std.610.12, software usability is the user's simplicity in learning to provide inputs and operate a given component or system [1]. Although software and system usability are considered as non-functional requirements, its importance cannot be overstated [2]. Sagar and Saha [3] listed six software quality criteria which includes usability as well. As modern-day software is evolving to be more complex and omnipresent, software usability has become an indispensable non-functional requirement for ensuring software quality.
In the field of software engineering, various studies have identified usability issues and its implication on software quality [4][5][6]. As per ISO/IEC 9126, software usability is the ability of users to generally understand, use, and learn the software with ease. Based on [7], there are five important usability attributes, which are learnability, attractiveness, understandability, operability, and usability compliance. Thus, the definition for software usability differs among different standards and researchers [3].
In the real world, various application domains have duly considered usability engineering as an important area. It is widely considered to be a far-reaching research area at large [3] and is used in domains such as aerospace [8], computer-based medical devices [9], defense [10], mobile devices [11], web applications [12] etc. Various research and review studies were conducted on software usability [3,[11][12][13][14][15][16][17][18][19][20][21]. However, based on our findings from the existing literature, we observed that a systematic mapping study (SMS) in this domain is lacking, confirming the claim of Bitkina,et al. [9] regarding the absence of UX/usability studies. Hence, this SMS is undertaken to fill this research gap through the extensive analysis of important studies that were published in the last decade (2011)(2012)(2013)(2014)(2015)(2016)(2017)(2018)(2019)(2020). Filling this research gap will help researchers and experts better understand software usability's efficiency, effectiveness, and development.
In an effort to fill the research gap and inclusion of significant works, this research followed systematic mapping methodology. This approach allows the research to capture key facts and details from literature using a well-defined process. To this end, an SMS protocol composed of search strategy, data extraction, selection criteria, and rejection criteria was formed. The main objective of this research is to investigate factors affecting software usability. Additionally, this researched also aimed at classifying the selected studies by knowing the existing contributions, research facets conducted, validation methods used, evaluation measures utilized, application domains, and lastly the overall demographics of the literature reviewed. The selected parameters will provide an overview of the general trends of the publications as well as the evolution of research in the domain of software usability.
In this study, the main contributions are as follows:  A detailed examination and synthesis of key studies on software usability.
 The study reviews primary studies (PS) and identify their distinct contributions.
 The mapping study analyzes the overall research productivity, demographics, trends, and challenges in software usability.
This research study is divided into six sections. Section II presents various aspects of software usability and discusses related surveys in the domain of software usability. Section III describes the research methodology including research questions, data acquisition and processing techniques. Results are presented in Section IV. Section V discusses the research findings as well as directions for future research, whereas threats to validity are discussed in Section VI. Section VII concludes the work.

II. SOFTWARE USABILITY AND RELATED WORK
This section discusses software usability by reviewing articles published in the area.

A. Software Usability
Software usability is a key characteristic of software quality. As per ISO/IEC 9126, usability is defined as "a set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users" [3,7]. Based on [7], software quality model is composed of six dimensions, which are functionality, portability, maintainability, efficiency, usability and reliability.
However, usability is inconsistently defined in the literature, where different standards and researchers have distinct definitions. In the following text, some definitions of software usability from various standards are outlined. In a study by Nielsen, the author defined usability based on five key dimensions. These dimensions are efficiency, errors, learnability, memorability, and satisfaction [22]. Efficiency means that the system should be efficient to be utilized by a user. When the system is efficiently utilized, productivity will increase. It is also expected that such systems have low error rates. Learnability is the ability to understand the system, and a good system is expected to have small learning curve. Memorability refers to the ability of the system to be easily remembered without requiring users to relearn things again. Lastly, satisfaction means that the user should feel gratified when using the system. Moreover, in the ISO 9241-11, usability is defined as the level where a product can be utilized by a given user in achieving a specific objective with efficiency, effectiveness, and satisfaction in a defined context [23,24].
In software engineering, usability issues in software directly or indirectly contribute to software quality problems. These problems often cause low efficiency and effectiveness. Based on this, users encounter various difficulties when using a specific software [25]. Further, the usability issues can also lead to low acceptance rate of software applications [26].

B. Related Work
This section summarizes the existing literature in software usability and identifies the research gaps.
Bastien [27] focused on the growing applications of Information and Communication Technology (ICT) based medical devices incorporating human-computer interfaces. The author highlighted that usability of these medical devices' hinges on their ease of usability. The author identified inspection-, user-and model-based evaluations as standard industry approaches for validating usability. The work considered various scenarios and provided directions for further research in usability testing for mobile applications. Usability testing can be performed in a number of manners such as questionnaires, interviews, automated testing etc. Sure [21] considered questionnaire as a method for usability testing where information about usability, user satisfaction, expectations, software behavior and other related information is collected using a set of questions. The author identified thirteen questionnaires from literature. The analysis of the questionnaires revealed that 'satisfaction' is the most common factor that is measured by the majority of the questionnaires.
Esmeria and Seva [18] presented a survey of works in relation to the usability of the websites. The authors argued that the process of usability testing should result in a usability index that can describe the usability of a website. A drawback of the work is the lack of rigor in selection of data sources. The authors relied on Science Direct and ACM Digital Library only and no specific query string was mentioned. Sagar and Saha [3] conducted a review of the software usability and reached to a variety of conclusions. The authors identified that so far researchers have not been able to unanimously agree on usability models. Further, the work highlighted efficiency, effectiveness, satisfaction, and learnability as standard measures used by a majority of the usability models. In terms of usability evaluation methods, questionnaires, usability testing and heuristic evaluations are the leading mechanisms.
Maramba et al. [28] conducted a systematic review of usability testing of e-health applications. The authors observed that although there is an exponential increase in the development and subsequence deployment of e-health applications, very few of these applications have published their usability evaluation results. The work further highlighted that questionnaire has been the de-facto method for usability analysis and argued that more qualitative and automated methods must be used in usability evaluation. Key limitations of the work include the limited time frame for the selection of potential works (April 2014 -October 2017) and the scope of the work which is limited to e-health applications only. Weichbroth [29] used Scopus as data source to conduct a systematic literature review of usability aspects of mobile applications. The author identified 790 relevant documents spanning 2001 to 2018. It was observed that the usability definition as given in ISO 9241-11 is used by majority of the works. The work identified 75 attributes associated with the usability of mobile applications and identified efficiency, satisfaction and effectiveness as most important considerations as indicated in the literature. Memorability, cognitive load and errors are identified as least considered attributes. Like Maramba,et al. [28], and Weichbroth [29] observed that controlled observations and surveys are mostly used in usability studies and suggested the use of eye-tracking, thinking loud, and interviews to be used in the future usability studies. Besides these studies, other important works include Coursaris and Kim [30], Harrison,et al. [31], and Quiñones and Rusu [32]. 1 Harrison, et al. [31] Table I presents a summary of the selected related works. Note that some works such as Bastien [27] and Coursaris and Kim [30] are not included in the table as these works did not use the standard methodology for conducting systematic literature review.
Despite the usefulness of these studies, none of the highlighted studies in this section conducted a systematic mapping study for software usability. Most papers study selection process is also arbitrary with no rigor or repeatability. Hence, we observed that there is no study in the software usability that categorize and analyze existing research with respect to their research facets, contribution facets, publication forums/trends, citation impacts, and so on. Thus, the aim of this research is to fill these gaps in the field.

III. METHODOLOGY
Systematic mapping studies (SMS) are conducted to provide a general overview of a research area by systematically classifying the existing works and identify the contributions of researchers in the area of study. The studies (SMS) largely explore current literature to examine the reporting of areas, publication frequency, research trends, and publication venues where the primary studies are published [33,34]. There are many shared characteristics between a systematic literature review (SLR) and systematic mapping studies. Some of the characteristics are the use of evidence-based searching and study selection procedures. However, SMS has a distinct objective from SLR and took a different approach to data analysis. SMS is primarily aimed at mapping and structuring an area of study. Hence, in this study, the SMS follows the general guidelines suggested by Petersen, et al. [34] and Kitchenham and Brereton [35]. Consequently, this study follows the pathway of similar studies that adopt these guidelines [36][37][38]. Fig. 1 outlines the process for our study. As depicted in Fig. 1, the SMS process comprises of five distinct stages. The initial stage is to define key research questions (RQs). In stage two, the search process is conducted by specifying the search terms for retrieving the primary studies (PS). In the third and fourth stages, the retrieved studies are screened to remove unnecessary and irrelevant studies. Lastly, the data extraction is conducted, and the systematic maps of the study are created. These stages are defined in the following text.

A. Research Questions
In this section, the research questions (RQs) of the study are presented with respect to our main research objective. The main objective of this SMS is to classify the selected studies by knowing the existing contributions, research facets, validation methods, evaluation measures, application domains, and the overall demographics of the selected studies. The key question of this SMS is "what is the state-of-the-art in software usability studies?". Based on the objective of the work, the key question is divided into six distinct RQs as presented in Table II.

B. Data Sources
A SMS heavily relies on the selected Primary Studies (PS). To achieve a good mapping, it is vital that the process of selecting the PS is conducted carefully. The literature on software usability was collected from 2011 to 2020. Six data sources were selected for our literature search. The data sources include ACM Digital Library, IEEE Xplore, Google Scholar, Science Direct, SpringerLink, and Taylor and Francis. www.ijacsa.thesai.org It is important to mention that the selected sources include most the publications in the area of usability. Scopus could not be included as the author was unable to obtain access to the restricted (subscription based) database of Scopus. However, the selected six sources for data retrieval are broad and comprehensive enough to provide coverage to most of the reputable publications' outlets. In addition, the selected sources are used in other studies such as Esmeria and Seva [18] and Maramba,et al. [28]. Table III highlights the data sources with respect to the studies identified in our initial result search. The initial search resulted in 9,874 studies, out of which 62 studies were shortlisted based selection criteria covered in Section 3.D.

RQ1
What are the demographic characteristics of the PS?
To identify publication trend, publication forums and citation impact of the primary studies.

RQ2
What contribution facets have the primary studies provided?
To identify the contribution facets (model, method, investigation, and so on).

RQ3
What are the research types (facet) focused on by the primary studies in the domain?
To ascertain the research facets (evaluation research, solution proposal, and so on) in the area of study.

RQ4
What validation methods are generally utilized for software usability evaluation?
To identify various software usability evaluation methods.

RQ5
What are the various application domains for usability?
To investigate the domains of application for usability.

RQ6
What are the evaluation measures used by the PS?
To identify the evaluation measures used by the selected studies.

C. Search Terms
We performed automatic searches to retrieve important studies from our selected data sources. This is achieved using our search string or terms developed based on the guidelines of Petersen, et al. [34]. Basically, a search string is a composition of characters used by a researcher to identify the most relevant set of documents from a data source. Therefore, selecting the right search string is imperative because the outcome is connected to the information given by the data source. Hence, the selection of right terms requires careful attention to ensure important studies are not missed in the mapping process. In doing so, a generic search string was formulated to work on all the data sources. The search string is outlined as follows.
((Software usability AND Usability models) OR (Usability metrics)) It is important to point out that the search term is relatively generic to ensure that maximum results are obtained. A carefully crafted inclusion and exclusion criterion is then applied to shortlist the publications for further study. After execution of the search terms on respective data sources, the resultant publications obtained from each of the data source are reflected in Table III. A total of 9,874 publications were retrieved. Note that the search query was executed to collect the raw data from 19 November 2020 to 25 December 2020.

D. Inclusion and Exclusion Criterion
When the search results utilizing the articulated search string are acquired, we anticipated that works that are not important to the objective of this SMS study can also be retrieved. If so happens, we cautiously designed a clear inclusion and exclusion criteria that will be used on the retrieved studies to eliminate those that are not in-line with the objective of the paper. The inclusion-exclusion criteria for this study are outlined as follows; After a careful implementation of inclusion and exclusion criterion, and thorough manual analysis including removal of duplicates, we identified 62 primary studies for further analysis.

E. Extraction of Data
To extract data from each of the 62 primary studies for answering the formulated RQs, a systematic data extraction method has to be clearly defined. We created a form to extract important data from the 62 identified articles for this study. The author as well as the two volunteers filled out the form for each of the 62 selected papers. For each publication, title, publication venue, research type, contributions, validation method, publication year, evaluation measures and application domain were recorded. www.ijacsa.thesai.org

F. Classification Scheme
We followed Petersen, et al. [34] to develop the classification scheme for this study. The 62 final selected studies were examined by their titles, abstracts, keywords, research contributions, theoretical models, and general demographics. These studies were comprehensively studied for a thorough understanding of various characteristics of the classification. The first step is to classify the PS into the contributions made by various researchers. These contributions are investigative study, evaluation study, model, framework, application, scheme, method, usability concepts, usability principles, approach, and system. We further extended the classification to identify the research facets. These facets are experience papers, evaluation research, solution proposals, and validation research. These classifications are standard and in line with the existing literature for conducting a mapping study [34,38]. Subsequently, further classifications of the PS were done with respect to the validation methods used to validate software usability, evaluation measures, application domains, and the selected studies general demographic characteristics.

IV. RESULTS
This section covers the results of the research. The RQs formulated are all answered by critically analyzing the PS studies. Table IV presents the PS for this study. For the sake of brevity, only research article IDs is provided. For mapping between IDs and research articles description, the reader is referred to Appendix A.

A. RQ1. What are the Demographics Characteristics of the PS?
In answering this RQ, the primary studies were analyzed critically with the purpose of answering the RQ. Three aspects of the PS were analyzed including publication trend, publication forums, and citation impact.
Publication trend: From 2011 to 2020, 62 studies were retrieved from the data sources. In Fig. 2, the year-wise publications in the domain of software usability are graphically presented. We observed that in 2013 and 2015, more studies were published with 12 and 9 studies, which are the most active years in the research domain. 2019 was also moderately active, with 7 studies. In general, even though the number of studies is linear, the research output continues to stabilize with stable yearly publication. Publication forums: This SMS study covered 20 different journals, 19 different conference proceedings, and 1 symposium and workshop each, respectively (see Appendix B). 39 papers are published in journals, 21 in conferences and one study each in workshop and symposium. From the analysis, we found that the International Journal of Human-Computer Interaction and the International Journal of Human-Computer Studies were the venues that contribute the most with 7 and 6 publications respectively followed by Journal of Systems and Software and Behavior and Information Technology with 4 and 2 publications each. Table V, the number of citations for the top 10 most cited papers is presented. We obtained the citation count of each paper from Google Scholar. Hence, the citation count may or will change at any point in time. In general, from our PS, we found four papers that have more than 100 citations each, which are Lee and Kozar [39], Kortum and Sorber [40], Mirkovic,et al. [41], and Maitama, et al. [36]. The total number of citations from the PS was 1809 as presented in Table IV. Therefore, the average number of citations per paper is 29.17.

B. RQ2. What Contribution Facets have the PS Provided?
To answer the RQ, we conducted a thorough analysis of the selected PS. Based on the analysis, we found 13 key contributions. These contributions are summarized in Fig. 3. The most significant contributions are Investigation (23 papers), Evaluation (18 papers), and Model (8 papers). 37% of the PS conducted an investigative study on software usability, followed by evaluation (29%), and model (13%), respectively. The rest of the contributions have less than 5% coverage. From our analysis, we observed that majority of the studies conducted an investigation into usability of existing web applications (B6, B15, B8, B38) or usability evaluation models (B37, B57, B53). Other studies focused more on proposing new models to help in facilitating or understanding key factors that facilitate or hinder usability of software or web application. Generally, we observed that a majority of the studies are not tailored to solution proposals, rather, they are focused on understanding software usability and ways to understand the factors that hinders users' acceptability and understandability of a software system. Despite the fact that software usability is not a recent research area, we observed that both investigative studies and evaluation studies are gaining attention from the researchers in this domain. This trend should be tailored into the proposition of new ways (in terms of framework, method, models, and so on) to help solve the usability issue in software engineering rather than just investigations and general evaluations. However, this is understandable because usability issues need to be understood using investigative and evaluation approaches. www.ijacsa.thesai.org  Fig. 4 shows the contributions on yearly basis. We observed that in 2018, no evaluation study was conducted. We further observed that from 2011 onward, research output is consistent and predictable. However, contributions such as framework, usability principles, usability guidelines, metric, system, usability concepts, method, scheme, and approach have received less attention in the field of study.

C. RQ3. What are the Research Types (Facet) Focused by the PS in the Domain?
From the PS, we identified four key research facets. These research facets are experience paper, evaluation research, solution proposal, and validation research. Experience papers are intended to give an explanation of how issues are handled in practice. Hence, it is generally the personal experience of the authors conducting the work. An example of the experience paper is Komiyama,et al. [42] where the authors attempted to obtain the developers point of view on the usability of intensive software system. Solution proposal seeks to provide new and novel solutions to an established problem in a research domain. The example of a solution proposal is Diaz, et al. [43]. In their work, the authors proposed a new usability metrics for ecommerce website. Evaluation research is conducted to understand how a method is implemented in practice. An example of evaluation research is the work of Al-Maani and Salameh [44]. Validation research papers present a novel proposal that is not fully implemented in practice. Examples of validation research are Störrle [45], and Christophersen and Konradt [46]. From our analysis, as presented in Table VI, we found that most of the studies conducted Evaluation research with 33% of the PS. Furthermore, experience paper constitutes 29% of the total publications, followed by solution proposal with 24%, and validation research with 13%. The analysis shows that there is an urgent need for more solution proposals and research to validate the proposed proposals. Fig. 5 depicts the map for the identified research facets in correspondence to the validation methods. We observe that more experience papers are needed in this research domain to understand users' perspective with respect to usability issues.

D. RQ4. What Validation Methods are Generally Utilized for
Software Usability Evaluation? Validation methods are key to evaluating one's work in any scientific work. In answering this RQ, we identify eight validation methods utilized by the respective PS. These methods are experiment (22 studies), theoretical validation (10), case study (5), questionnaire (3), interview (3), interview and questionnaire (2), simulation (2), and field study (1). We observed that experiment and theoretical validation are the most used approaches by the PS in this domain. In Table VII, the identified validation methods with respect to the studies that used them are highlighted.  In answering this RQ, we identified six application domains in the research area, which are web (17 studies), software development (16), mobile (8), industry and research (6), navigation (3), and robotics (1). We identified 11 studies that have no well-defined domain. Web is the most considered domain with 27% of the PS, followed by software development (25%). Table VIII presents the application domains in the research area.

F. RQ6. What are the Evaluation Measures used by the PS?
With respect to evaluation metrics, we observed that most of the studies (16) used usability as a metric. This is followed by effectiveness (4 studies) and learnability (4 studies). This research observed that some studies did not clearly identified the evaluation metric that were utilized by the research. This observation is quite alarming, because almost of the primary studies have no well-defined evaluation measures in their studies. Hence, this need to be address and more work need to clarify or use an evaluation metric for their study.

V. DISCUSSION
The discussion in this section comprises of two facets. Firstly, the main findings of the study are summarized and presented clearly. It is followed by highlighting areas in usability that have not received considerable attention from the researchers.
From 2011 -2020, the research in software usability is generally stable. Most of the PS are published in journals (39), followed by conferences (21), workshop and symposium (1 each). We found that International Journal of Human-Computer Interaction and the International Journal of Human-Computer Studies are the venues that contributed the most with 7 and 6 publications, respectively. With respect to contribution facets, multiple key contributions were identified. 37% of the PS conducted an investigative study on software usability, followed by evaluation with 29% of the PS, and model with 13%, respectively.
From our analysis, we observed that majority of the studies focused on usability testing of existing web applications (such as B6, B15, B8, B38) or usability evaluation models (such as B37, B57, B53). Other studies focused on proposing new models to help in understanding key factors that facilitate or hinder usability of software or web application. We observed that a majority of the studies are not tailored to solution proposals, rather, they are anchored in the direction of understanding software usability and ways to understand the factors that hinders users' acceptability and understandability of a software system.
From the PS, we identified four key research facets which are evaluation papers (33%), experience papers (29%), solution proposal (24%), and validation research (13%). We also identified eight validation methods utilized by the respective PS. These methods are experiment with 22 studies, followed by theoretical validation (10), case study (5), questionnaire (3), interview (3), interview and questionnaire (2), simulation (2), and field study (1). The research also identified three major evaluation metrics for evaluating proposals, which are used in this area. These metrics are usability, effectiveness and learnability. The proposals have been evaluated using a combination of metrics and sub-metrics as well. Some of the primary studies did not use any evaluation measure for validation. Hence, this needs to be addressed.
From our study, we observed that although a significant research is conducted to improve the usability of the software applications, there are still a number of research directions that needed to explore in more detail. For instance, when evaluating usability of software, majority of the works relied on closed experiments, questionnaires and surveys. It is recommended that future studies should focus more on field studies where an application usability is evaluated in real world conditions. Field studies are more beneficial as these are conducted by the end users in real operational environment resulting in identification of problems that might be overshadowed in laboratory-based tests. However, conducting the field-based study can be expensive both in terms of time and monetary value. An important aspect of software testing is the reproducibility of the errors/bugs. It is important to consider reproducibility of usability testing as a dimension in usability metrics.
Usability testing are used in a variety of domain such as aviation [47,48], banking [49], bioinformatics [20] and medicine [28]. One area that is neglected by the researcher is the usability of software used by the scientific community. Most of the software applications developed for the use of scientific community are either command line or has poor user interfaces and faces numerous usability challenges. It will be important to consider usability testing of the scientific software and design a set of standard guidelines.
Ethics is an important aspect of software engineering process [50], however ethics in usability testing has received the least consideration by the researcher. It will be of interest to examine compliance of various ethical guidelines in usability testing. The ethical dimension of the usability testing is more important when considering the applications in domains such as health and finance.
Although considerable efforts are devoted in the literature to investigate the usability of mobile applications, we could not find any studies that focused on the cross-platform usability of www.ijacsa.thesai.org various mobile applications. Generally, mobile applications are developed for different platforms (notably android and iOS). It will be helpful to investigate how the usability of mobile applications varies across different platforms.
In the recent past machine learning applications have seen widespread use in various domains [51,52]. However, there is limited work to evaluate the usability aspect of machine learning applications [53]. As the machine learning applications are adopted for more widespread use, it is recommended to conduct usability testing of these applications. Although, open source software has gained traction and acceptance, there is little work to assess the usability of such systems [54]. It will be important to assess the usability of open source software systems.

VI. THREAT TO VALIDITY
After the rigorous analysis of the primary studies, this section discusses some identified issues that can be regarded as threat to validity. These limitations are discussed as follows.
Selection bias can be regarded as an external threat to the validity of our study. With respect to selection bias, even though the selected data sources were thoroughly searched by the author, there is a possibility that some important studies might be missed. To reduce this bias, this study employs inclusion and exclusion criteria for the selection of quality papers. The study is conducted by a single author which might result in selection bias. In order to mitigate the effect of the personal selection bias, two volunteer researchers oversaw the process and provided the feedback on the selection of final research papers for the study. Therefore, the threat level of personal selection bias is mitigated.
Conclusion validity is minimized by drawing all the relationships and conclusions from the literature based on the search query and then analyzing them using statistics such as citation count, classification into various sub-fields etc. Publication bias was mitigated by searching six important data sources. We also employed forward and backward snowballing techniques to make sure that all relevant studies related to software usability is identified, properly vetted and considered.
With respect to misclassification, some primary studies showed a limited information. Hence, some information was inferred while other where classified as "Nil" during the classification process. Hence, the inadequacy of information during classification may result in bias. In such a situation, the general methodology of a given study is carefully considered including the experimental setup to infer other classification entities. Therefore, this threat is mitigated to a certain level.
The selection of keywords can be regarded as threat of construct validity. We selected generalized keywords to identify a larger set of research papers in our domain from the various databases. Although it helped in reducing the probability of missing a relevant article, it has resulted in a large number of hits. In order to ensure that only relevant papers are selected for the study, a careful approach involving two volunteers as well as a carefully crafted approach is employed ensuring selection of relevant papers for the study.
Finally, the data was retrieved from the six selected data sources from 19 November 2020 to 25 December 2020. Therefore, if the same search query is executed at any later stage, different results might be obtained. Likewise, the citation count can be different as well as the research articles might have accrued more citations since 25th December 2020.

VII. CONCLUSION
This work presented a mapping study that analyzed research work from 2011 to 2020 in the domain of software usability. From an initial pool of 9,874 papers, 62 papers were carefully selected based on our inclusion/exclusion criterion. This study examined the existing contributions, research facets, evaluation measures, validation methods, application domains, demographics, publication trends, and publication forums in the research domain. With respect to contributions, investigative studies and evaluation studies are the two most common approaches. We identified four key research facets, which are experience paper, solution proposal, evaluation research, and validation research. We also identified eight validation methods utilized by the respective PS. Usability, effectiveness and learnability are found to be the common evaluation metrics. Rather alarmingly, some of the primary studies have no clear evaluation metrics defined. Hence, this need to be address and more work need to clarify or use an evaluation metric for their study.
In conclusion, the aim of this mapping study was to allow researchers and experts to have a clear understanding of the general research productivity, trends and demographics that shaped the research domain of software usability. This work will help in highlighting potential opportunities for both new and experienced researchers to conduct more works with the aim of improving the research domain.