Evaluating Web Accessibility Metrics for Jordanian Universities

University web portals are considered one of the main access gateways for universities. Typically, they have a large candidate audience among the current students, employees, and faculty members aside from previous and future students, employees, and faculty members. Web accessibility is the concept of providing web content universal access to different machines and people with different ages, skills, education levels, and abilities. Several web accessibility metrics have been proposed in previous years to measure web accessibility. We integrated and extracted common web accessibility metrics from the different accessibility tools used in this study. This study evaluates web accessibility metrics for 36 Jordanian universities and educational institute websites. We analyze the level of web accessibility using a number of available evaluation tools against the standard guidelines for web accessibility. Receiver operating characteristic quality measurements is used to evaluate the effectiveness of the integrated accessibility metrics. Keywords—web accessibility, web ranking, web evaluation, web testing


INTRODUCTION
University webmasters enhance their methods and techniques to present their content in the best possible way and to enable users to access and satisfy their needs easily.Search engines use many factors that affect web accessibility, thereby allowing the enhancement and improvement of these factors to further improve the websites' level of appearance in the search engine results page (SERP).Furthermore, university websites require obtaining the highest possible position in the SERP to increase their number of users (depending on web search engines such as Google, Bing, and Yahoo) who search for information.Web accessibility and search engine optimization share many features, such as using keywords in the following HTML tags: headings (e.g.<h1>, <h2> etc.), page titles <title>, anchor texts <a>, and alt attributes on images, which play an important role in improving the ranking of websites.Ivory et al.'s [1] study indicates that results of web search engines for blind users should be different from the results dedicated to normal users because the relevance of blind users is different from that of normal users relative to a certain query.Therefore, web search engines must consider web accessibility as a factor affecting their ranks for disabled web search engine users.Google Inc. launched in 2006 an experimental search engine called Google Accessible Search Engine (http://www.google.com/accessibility/labs/search/),but this search engine is no longer supported by Google Inc.An observational study on search behavior is conducted by Sahib et al. [2] to design and implement more accessible and usable search interfaces for both visually impaired and sighted web searchers.They further propose in their study the necessary design guidelines for interfaces to make them usable and accessible to screen readers.Using web accessibility within commercial websites by webmasters may enable more people to visit these websites and, in most cases, more customers (more sales).Furthermore, adopting web accessibility by webmasters means that websites that will be easier to maintain, update, and redesign and different devices will be improved.Web accessibility has a positive effect on search engine optimization (SEO).Moreno and Martinez indicate in their study that accessible web pages regularly appear within the top of SERP without applying SEO techniques because of similarities and overlapping characteristics between many SEO factors and web accessibility guidelines [3].Commercially, decision makers consider struggling for accessibility does not bring return on investment, that is, the costs of ensuring accessibility are larger than the revenue provided by new customers.Independently, adopting web accessibility does not mean easy maintenance upgrade and design.
Web accessibility means making the web more accessible to people with different abilities.Furthermore, different machines, including assistive technologies, can be used to access the web.Web accessibility enables disabled people/people with special needs to use the web, thereby making web accessibility an important issue in web development.The disabilities affecting web access includes visual (blindness, Kalnienk vision, and low vision), auditory (hard of hearing, deafness, and deaf-blindness), speech (low speech, high speech, stuttering problems, influent, articulation problems), physical (arthritis, Parkinson's disease, essential www.ijacsa.thesai.orgtremor, multiple sclerosis, broken arm), learning (emotional disturbance, intellectual disability, dyslexia), cognitive, and neurological disabilities.Web accessibility is defined as the ability of users to universally access websites and obtain their information needs.According to the World Health Organization (WHO), over a billion people (approximately 15% of the world's population) have some form of disability [4].Thus, this issue is an important one on web accessibility for this portion of the world population.
Web accessibility overlaps with but is not the same as usability, and it is different from device independence.A number of individuals and organizations have proposed various standards and guidelines for content accessibility throughout the web's history.Only the Web Content Accessibility Guidelines (WCAG) have been reviewed by the World Wide Web Consortium (W3C) members, software developers, and other W3C groups and interested parties, and are endorsed by the W3C director as web standards.The goals of web accessibility guidelines have to be achieved to assist us in developing and evaluating web accessibility.WCAG 1.0 was released on May 5, 1999 and WCAG 2.0 was released on December 11, 2008 to assist web programmers to build web pages accessible to disabled people and web content accessible from different environments or platforms.They later became an ISO standard.WCAG 1.0 and WCAG 2.0 were proposed by W3C (www.w3.org).Each WCAG 1.0 guideline is divided into checkpoints, which indicate good practices for constructing accessible web content and assist web developers in avoiding barriers that may prevent users from accessing the web.Compared with that of WCAG 1.0, the scope of WCAG 2.0 is broadly applicable to modern and future web technologies.Furthermore, WCAG 2.0 can be evaluated more accurately, manually, and automatically.In other words, it is more testable than WCAG 1.0.International efforts contribute to the production of WCAG 2.0 to harmonize a single standard for web content.WCAG 1.0 and WCAG 2.0 are currently the most broadly accepted qualitative measures for web accessibility [5][6].
Universities, as an essential part of the world of academia, have to be universal and include all disabled people.Therefore, their websites have to be universal and accessible to all web users.Many studies have focused on evaluating the web accessibility features of many websites.However, we observe that web accessibility plays an important role in enhancing the ranking of websites.A study by Schmetzke finds that only a few American university websites are accessible [7], but later studies show that most of these websites are adopting accessibility policies.In this study, we aim to discover the accessibility of Jordanian university websites by analyzing and evaluating the metrics of web accessibility as a case study.
Vigo et al. [8] find that evaluations based solely on automated web accessibility tools are unreliable in terms of finding all errors or reporting errors that do not exist.Nevertheless, these tools are used in this study to analyze extracted web accessibility features using various tools and to find the common main shared features among the different tools.In the case study, we use selected features to evaluate web accessibility levels in a case study of university websites and higher educational institutes in Jordan.We further evaluate these features using four machine learning classifiers.
The rest of this paper is organized as follows.Section 2 presents the literature review and discusses the issue of web accessibility metrics for Jordanian universities.Section 3 discusses the methodology, and Section 4 presents the experiments and results.Section 5 outlines the main findings of this study and the planned future works.

II. RELATED WORK
Recently, interest in measures of web accessibility has grown, and thus the literature has witnessed a substantial increase in the number of web accessibility metric studies.This section presents the previous studies on web accessibility metrics using different methods.The last part of this study presents some studies conducted on Arabic websites.
The first published study on accessibility measurement is that by Sullivan and Matson [9].In their study, they mention that 95% or more of all websites are inaccessible and that it is a huge problem facing disabled users of the web.This finding indicates a clear ignorance to the issues of universal design and content accessibility.Therefore, they conduct a content accessibility compliance audit of the top 50 websites' most highly trafficked sites according to Alexa.com to determine whether these websites are accessible or not.They use a Lift Online tool in their study and determine if a substantive relationship exists between content accessibility and usability through the Spearman rank-order correlation coefficient.The statistical analysis shows that a weak relationship exists between content accessibility and overall usability [9].They use the failure rate metric that considers the concept of "potential problems," and this metric for a given web page simply represents the ratio between the total number of real errors in a web page and the total number of potential errors.Furthermore, they use another metric that penalizes web pages with a large number of elements that cause accessibility errors.This metric is calculated by multiplying the number of accessibility opportunities (potential points of failure) by their failure rate.Penalizing accessibility opportunities is caused by the probable inclusion of accessibility barriers.
The Web Quality Evaluation Method (WebQEM) was proposed for the first time by Olsina [10] and then used by Olsina and Rossi [11] to evaluate and compare quality requirements for websites and applications.González et al. [12] adopt another important accessibility metric that considers the concept of "weight" for barriers.They aim to enhance the accessibility of web pages for visually impaired web users.This important metric is based on the WebQEM model, and a global ratio is calculated for a given web page.This global ratio is later multiplied by a weight, which is defined according to the effect of each barrier.The concept of barrier weight coefficients was first proposed by [12], and thus, later metrics used this concept.
Parmanto and Zeng propose another quantitative metric called the Web Accessibility Barrier (WAB) to measure the content accessibility of different web pages for disabled web users [13].WAB considers the concept of potential problems and weights for barriers.The WAB metric considers the size www.ijacsa.thesai.org(total number of pages) contained in a given website.A high WAB score indicates that barriers exist, and a low WAB score indicates that the website under study complies better with the WCAG guidelines.A WAB score of zero means that the website under study has no barriers.The study conducted by Hackett and Parmanto [14] refers to the WAB score as a proxy of web accessibility and concludes that WAB is unable to differentiate between barriers posing minimal limitations and those posing absolute inaccessibility.
The Unified Web Evaluation Methodology (UWEM) was developed in Europe by the Web Accessibility Benchmarking (WAB) Cluster to be mainly used there.It was developed to become a standard for evaluating web accessibility.UWEM is a completely automatic accessibility metric; therefore, it provides its users with suitable methods and advice to carefully evaluate a set of websites or a single website.For a single web page, the UWEM final value depicts an approximation of the probability of discovering a barrier in a website that could prevent a user from finishing a specific task [15] [16].In their study, Buhler, Heck, Perlick, Nietzio, and Ulltveit-Moe [15] observe that WAB does not support different disability groups.Furthermore, web pages with a low number of various barriers are considered by WAB to be more accessible, and this aspect is undesirable.Accordingly, they propose a new aggregation metric (A3) to adapt the measurement to different disability groups, and this metric represents an improvement of UWEM 0.5 [15].The A3 metric is similar to other metrics on the verification of checkpoint conformance.It uses some probability properties and aggregated some issues related to the complexity of the web page under consideration.The A3 metric considers the number of violations of a given checkpoint in relation to the total number of violations [15].
The study of Vigo, Arrue, Brajnik, Lomuscio, and Abascal [17] shows the importance of quantitative accessibility measurements and proposes the three different applications: information retrieval, quality assurance within web engineering, and accessibility monitoring.They propose an automatic quantitative metric to evaluate accessibility called Web Accessibility Quality Metric (WAQM) based on the reports of automatic evaluation tools.Fifteen websites (1363 web pages) and two automatic evaluation tools (EvalAccess and LIFT) are used to verify the reliability of their proposed metric.They conclude that their metric results are highly dependent on evaluation tools, and a high correlation exists among the results of different tools.Therefore, they deduce that their metric can be used by information retrieval systems to rank results, and that this metric is beneficial for accessibility monitoring scenarios and partially beneficial for web engineering scenario [17].A total of 918 web pages belonging to 10 European, United States, and African university websites, as well as 445 web pages belonging to five newspaper websites, are utilized in their tests.
Freire, Fortes, Turine, and Paiva's study [18] reviews six web accessibility metrics used in previous years and compares them.They discuss the strengths and pitfalls of these six web accessibility metrics.Therefore, they present the first known web accessibility metric (Failure Rate) [9].Buenadicha et al. [19] examine accessibility as a subcategory of web assessment index, which includes the following four categories: speed, navigability, and content, apart from accessibility.Through a detailed literature review, these authors identify the key factors considered as determinants of website quality and use them in their index to evaluate all websites of Spanish universities.
Kane et al. [20] present an evaluation of the previous state of the university website accessibility of the 100 top universities' home pages worldwide.They analyze the compliance of these 100 home pages with image accessibility, accessibility standards, text-only content, quality of web accessibility statements, and alternate language.Their study is limited to and based only on the analysis of only 100 web pages.The results of their study [20] show that many top universities have accessibility problems, and a significant variation in accessibility exists among these universities across different countries and geographic regions.This study [20] concludes that the accessibility of websites of universities in non-English-speaking countries is either low or does not exist.
Many accessibility studies include US higher education establishments, such as those conducted by Harper et al. [21], and aim to raise awareness on accessibility issues in higher education websites.These authors invited webmasters of higher education institutions to evaluate the overall accessibility of their websites using freeware.Study [21] indicates that most of the university homepages under study were non-compliant with the WCAG, and that only one establishment satisfied all W3C guidelines and gained a Triple A. Bradbard et al. [22] examine the accessibility of 58 wellknown US universities.Data from Peterson's Four-year Colleges (2007) on these top 58 US universities were used in their study.Results show that only 50 of these universities adopt accessibility policies and that 78% and 88% of these universities neglect the timeframe to implement their policies and violate these standards, respectively.According to [22], only two US universities (Purdue University and University of California) have good accessibility policies that could serve as models for other universities worldwide.
Accessibility in higher education institutions is not restricted to websites as it also includes Learning Content Management Systems (LCMSs).Therefore, a number of webbased open-source LCMSs have been explored by researchers.One of these studies was conducted by Iglesias et al. [23] to evaluate three web-based open-source LCMSs (ATutor 1.6.2,Moodle 1.9.4,and Sakai 2.6.0).This type of study requires an assessment and monitoring of LCMS accessibility to guarantee the universal accessibility of this type of systems.Similar studies were conducted by [24], [25], and [26], among others.

III. METHODOLOGY
A convenience sample of Jordanian higher education websites was studied for the year 2015.In this study, we used various web accessibility tools to analyze the web accessibility metrics for the Jordanian university websites and to evaluate their level of web accessibility.The following steps describe the methodology of this study: www.ijacsa.thesai.org 1) Select a number of popular web accessibility tools according to the W3C (http://www.w3.org/WAI) guidelines, which provides various recourses belonging to web accessibility guidelines, tools, and standards.
2) Apply the selected tools on Jordanian universities websites as a case study.
3) Extract a number of web accessibility features and metrics, and find common and shared features among all the tools.
4) Evaluate the selected shared metrics using the receiver operating characteristic (ROC) quality measurements.ROC is an essential evaluation of prediction metrics used to identify the possible best selected metrics.
A. This study is based on a dataset that consists of 36 Jordanian universities, including 9 public universities and 27 private universities and institutes.Each university website under study is represented by the two most visited web pages, namely, the homepage and the registration web page (for a typical university website).The total available web pages are 72.B. Accessibility guidelines present the methods to improve website visibility for users.We select the following seven free web accessibility tools to be used in this study: 1) A Checker tool: This tool is used to assess web accessibility problems for the HTML content, where the complete HTML source code for each web page under study is www.ijacsa.thesai.orgpasted.AChecker tool divides problems into the following three main parts: Known, Likely, and Potential problems [27].Table II presents the web accessibility guidelines (WCAG 2.0 [Level AA]) for this tool.
2) Cryptzone Cynthia Says is a joint education and outreach project of Cryptzone portal and the Internet Society Disability and Special Needs Chapter.It is used to evaluate and provide feedback on website accessibility according to the US Access Board's Section 508 or the W3C's WCAG 2.0 A-AAA Accessibility Guidelines [28].
3) Functional Accessibility Evaluator (FAE) is developed to assess web pages' accessibility according to the W3C Web Content Accessibility Guidelines 2.0 Level A and AA requirements using the accessibility features and techniques associated with W3C ARIA 1.0 and HTML5 specifications [29].
4) HERA is designed to evaluate web pages' accessibility according to the specification WCAG 1.0.HERA applies a set 5) of tests on the web page and finds any automatically detectable problems, which require further manual verification [30].

6)
Validator tool from the W3 organization is used to evaluate the mark-up validity of web documents in HTML, XHTML, SMIL, MathML, and so on [31].
7) Wave WebAIM tool is used to assess and enhance web developers to improve the accessibility of their web content.It is easy to use by simply entering a URL and waiting for the results [32].
8) TAW is an online free tool used to evaluate website accessibility according to the W3C WCAG 1.0 [33].

IV. EXPERIMENTS AND RESULTS
In our experiment, we applied these seven free online tools on Jordanian university websites.Thus, for every website, we tested the two most visited web pages, namely, the homepage and the registration web page, which are considered highly visited web pages by users.We combined the features for every tool and found the common features among all the tools.Then, we summarized the values for each university.The dataset was divided manually into three main categories based on the optimal and worst results (high, medium, and low).We evaluated the selected metrics using ROC quality measurements to assess the effectiveness of the common extracted features.The HERA tool provides detailed results for every point in the results.The green tick symbol means that HERA has a positive reflection of applying web accessibility, the x-mark refers to missing web accessibility metrics, and the gray tick symbol refers to inapplicable metrics.Verify if the images using alt element or keep it empty.
2. Distribute the color.The harmony between using font and the background.

Web page title element.
The title should provide description for the content of the Web page.4. Navigate to relevant websites.
Check whether that the Website points to another universities.

5.Using of Heading elements.
There is a balance needed to using headings tags on a Web page, to provide useful structure and outlines to users.
6.Primary language.Language tags use a primary code to indicate the language such as: language and XML: language attributes.Furthermore, optional sub codes to indicate variants of the language.
7. Using submit buttons.Provide a technique that allow users to explicitly request changes of context, such as: generate an http to submit data that entered in a form.8. Labels description.
Ensure that the label for any interactive component within Web content makes the components aims transparent.9. Enough Time.
Provide users enough time to read and use content.We used these features that we consider the main web accessibility metrics to rank our dataset.We divided the data into three main categories based on the optimal and worst results (i.e., high, medium, and low).Table IV shows the ranking of universities based on the combined features.Then, we used the ROC quality measurements to evaluate the effectiveness of the following selected features after the 10fold cross-validation process: True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN), Precision, Recall, and F-Measure (F-M).The ROC represents a graphical plot that illustrates the performance of a binary classifier system [35].Formula 1 represents the Accuracy formula Table V presents the experiment results for the selected common features using the K nearest neighbor (K-NN) when the k = 1 classifier yields an accuracy of 88.9%.The detailed results for the SVM show high accuracy results for the high class, medium accuracy results for the medium class, and very low accuracy results for the low class.Thus, the overall weighted average results yield an accuracy of 61.11%.
Table VII presents the results for the selected common features using Decision Tree (J48).J48 yields an accuracy of 94.4% and an error rate of 5.6%.Table VII shows that J48 yields better results than the previous classifiers K-NN and SVM.
Table VIII presents the results using a bagging classifier, which splits the dataset into many sub-datasets.The bagging classifier computes the prediction in each sub-dataset, selects the most frequently predicted results, and finally considers them as the final dataset prediction [35].Table VIII indicates that the bagging classifier yields an accuracy of 97.2% and an error rate of 2.8%, with the low class providing the optimal results for all the measurements.The bagging classifier shows better results than the three previous classifiers.
Finally, Table IX   The results in Table IX indicate that the bagging classifier yields the best KS effectiveness measure results for the selected features, whereas SVM is the lowest classifier with KS value.

V. CONCLUSION AND FUTURE WORK
Websites are important especially for higher educational institutes.They are considered as the main gateway to the world.The academic ranking of a university in general can be dependent on the quality of its main website and its ability to provide relevant information and services to users.Web accessibility is considered one of the major important quality goals in designing websites in particular and software applications in general.It ensures that a developed website can be equally accessed by a large category of users regardless of www.ijacsa.thesai.orgtheir physical abilities, skills, locations, languages, backgrounds, and so on.
In this paper, we evaluated most of the websites of Jordanian universities in terms of accessibility.The results showed a significant number of weaknesses in most of the universities.Furthermore, a variation of web accessibility standards was found when the websites were measured using different accessibility tools.
Fig. I shows part of the homepage of Zarqa University using the AChecker tool.

Fig. 1 .
Fig. 1.First GUI of A-Checker tool Fig. II shows part of the Yarmouk University homepage results using the Cryptzone Cynthia Says tool.

Fig
Fig. III presents the FAE tool that defines the number of rules according to web accessibility guidelines, such as heading, tables, links, and styling content.The FAE tool provides detailed results for every rule.Fig. IV illustrates the HERA results of the German Jordanian University.

Fig. 3 .Fig. 4 .
Fig. 3. Summary of the results for the Jordanian University using the FAE tool Fig. V presents part of the results for Al Al-Bayt University using the Validator tool.

Fig. 5 .
Fig. 5. Results for the Al Al-Bayt University using the Validator tool Fig. VI illustrates part of the results for the World Islamic Sciences and Education University using the Wave WebAIM tool, which presents a summary of errors, alerts, features, and the structural element.

Fig. 6 .
Fig. 6.Summary of the results for the WISE University using the Wave WebAIM tool Fig. VII.Presents part of the results for Al Quds College using the TAW tool.

Fig. 7 .
Fig. 7. Summary of the results for Al-Quds College using the TAW tool The TAW free online tool presents a summary of the results as shown in Fig. VII.Through this tool, we can obtain the detailed results for every note.Applying the seven tools on

Formula 2 Formula 4
represents the Recall formula [55]: represents the F-Measure formula [55]: www.ijacsa.thesai.org Table I lists the names of the university websites under study excluding the type of university, whether it is public or private.

TABLE I
Table III presents the combined features for web accessibility.

TABLE III .
COMBINED FEATURES FOR WEB ACCESSIBILITY [72 -34]Feature Description 1. Images using alt element.
10. Readable.Make text content readable and understandable.11.Using anchor tags.This allow the users to navigate in large Web pages easily.12. Image links.Ensure that the image link missing or available.13.Web page size.Size in Kilobytes.14. Content Visibility.The visible page fraction inside the <page> element, against hidden text inside a specific Web page.www.ijacsa.thesai.org15.Compress.The total size of compressed files inside a specific Web page, and the total size of compression ratio inside a specific Web page.16.Images.Total number of Images inside a specific Web page.17.Image size.Total Image Size.18. Structural elements.Ensure structural elements such as: scroll, section, header, footer, article, and aside elements, whether missing or available.19.Style sheets.Style sheets code is correct or not.20.Broken link.The total number of broken links in Web page.21.Empty link.Total number of links without anchor text, and anchor text without links within the Web page.22. Redirected link.The total number of redirected links in Web page.

TABLE IV .
RANKING OF UNIVERSITIES USING THE COMBINED FEATURES

TABLE V .
DETAILED RESULTS FOR K-NN

TABLE VI .
DETAILED RESULTS FOR THE SVM

TABLE VII .
DETAILED RESULTS FOR J48 presents the effectiveness measurements of the four previous classifiers, namely, Kappa Statistic (KS), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Relative Absolute Error (RAE), and Root Relative Squared Error (RRSE).

TABLE IX .
DETAILED RESULTS FOR THE BAGGING CLASSIFIER