An Accessibility Evaluation of the Websites of Top-ranked Hospitals in Saudi Arabia

Hospital websites offer the potential to improve healthcare service delivery. They can provide up-to-date information and services to patients, at low cost and regardless of their level of abilities. This, in turn, can reduce overcrowding in hospitals and reduce spread of disease, especially in circumstances like the current COVID-19 pandemic. It is, therefore, imperative for designers to ensure the accessibility of hospital websites to the widest possible range of people. This study aims to evaluate the accessibility of the websites of topranked hospitals in Saudi Arabia using AChecker. The sample included the websites of the top ten hospitals from each of the public and private sectors. The results show that only 20% of the evaluated websites conformed fully to the Web Content Accessibility Guidelines 2.0. No significant difference was found in terms of the accessibility compliance between the websites of the public and private hospitals. The most frequently observed accessibility errors were related to the structure of information, non-text content, labels and instructions, headings, and keyboard access. The study concludes that Saudi hospitals are not doing an adequate job of meeting accessibility guidelines, thereby denying many of their web customers the ability to fully use their websites. Keywords—Accessibility; hospital websites; Saudi Arabia


I. INTRODUCTION
The use of information and communication technologies (ICT) to provide services related to health, known as e-health, has been introduced to facilitate equal access to healthcare services to all patients regardless of their abilities and disabilities, and to reduce healthcare delivery costs, hospital overcrowding and spread of diseases [1]. Indeed, people tend to use the Internet to find a wide range of information related to health, for consultations with health practitioners, and to participate in support groups [2]. According to a study of trends in consumer health, four out of five adults with an Internet connection use digital health technology in some form [3]. However, whilst there is valuable healthcare information on the Internet, there are also potential dangers from outdated, inaccurate, and even harmful medical information on-line [4].
An important quality attribute of hospital websites is accessibility, which directly impacts the use of websites by disabled people. In other words, web accessibility is the process by which interfaces are made more user-friendly and inclusive, and capable of operation by any user, in varied situations and under diverse conditions. If hospital websites are not accessible, patients with disabilities may have difficulty finding and understanding health information [5].
Hence, evaluating the accessibility of hospital website is critical.
The healthcare system in Saudi Arabia is one of the largest and most complex sectors. It serves a diverse population, of varied socio-economic backgrounds and in different geographical settings. Furthermore, according to Census of Saudi Arabia in 2017, 7.1% of the total population are suffering from some disability [6]. It is, therefore, important to have hospital websites that are accessible to a wide range of users. The current study examines the accessibility of the websites of the top-ranked hospitals in Saudi Arabia. The study has numerous implications related to hospital website accessibility. It is believed to be the first evaluation of the accessibility of Saudi hospital websites. The findings will be helpful for the Saudi hospital policy-makers such as ministry of health, hospital top management, in designing the policies and programs on e-health success in the country.
The structure of this paper is as follows: Section II reviews the existing literature and states the research questions of the study. Section III presents the evaluation methodology used. Section IV presents and discusses the results obtained. The final draws the conclusions of the study.

II. RELATED WORK
Several studies have considered the evaluation of hospital website accessibility worldwide. Mira et al. [7] assessed and compared hospital websites from and the USA, Great Britain and Spain in terms of ease of use, accessibility, and content quality. The results of their study showed that only ten of 32 websites analysed met accessibility standards. Kurniawan and Zaphiris [8] analysed health websites using an accessibility evaluation tool called "Bobby", which only approved 28% of them. The researchers emphasised that government websites must be subject to stricter government regulations with the U.S. Telecommunications Act, Section 255, which requires websites of US Federal agencies to be as accessible and usable as possible.
A 2009 study evaluated 53 websites from public-sector hospitals in Greece and found that most lacked key features of good practice in site accessibility [9]. Another study performed in 2010 examined the application of the Korean Web Content Accessibility Guidelines (WCAG) on the websites of 80 hospitals. It found that while many hospitals had attempted to achieve web accessibility, success required the aggressive implementation of the guidelines [10]. A 2016 study of Catalan private hospitals evaluated their websites" www.ijacsa.thesai.org characteristics and quality in terms of usability, accessibility, interactivity, content, quality references, content updates, privacy policies and web 2.0. The results indicated that accessibility evaluations are critical in the development of a website [11]. An additional study of the usability, accessibility and security of Indian hospital websites concludes that evident accessibility problems could be explained by the lack of application of the WCAG 2.0 standard [12].
In 2017, another study investigated the factors influencing the quality and visibility of the websites of private hospitals. The results showed the importance of presenting information with quality-accredited web accessibility [13]. O"Grady assessed 49 Canadian healthcare websites with WCAG 1.0 Priority 1 level. Only approximately 40% of pages examined were free of errors [14]. A study of 108 sites related to general health, by Zeng and Parmanto, found that none of them satisfied the WCAG Priority 1 guidelines [15]. A study in 2014 assessed the websites of 2785 hospitals in the USA, concluding that they lacked accessibility, scoring an average of 5.08 out of a possible ten [16]. Llinás et al. carried out a qualitative study of hospital web-portals. Of the 32 sites studied, twelve were Spanish, ten British and ten American. Only ten met the specified accessibility criteria [17]. A study by Maifredi et al. considered every Italian hospital that had a functional website. Quality ranged widely and most had severe limitations [18]. Mira et al."s study of the accessibility and readability of Spanish hospital websites revealed that none satisfied accessibility requirements [19].
Whilst there has been an increase in the use of the Internet and particularly for health purposes in Saudi Arabia [20], no study has been carried out to assess the accessibility of the websites of Saudi hospitals. It should be mentioned that the Government of Saudi Arabia has established legislation covering disability in terms of skills development and employment, although the specifics of web accessibility are not addressed [21].

A. Web Content Accessibility Guidelines
The WCAG 2.0 includes a range of recommendations designed to improve the accessibility of website content. This is for the benefit of a people with a broad spectrum of disabilities, individually or in combination, such as visual (reduced sight, blindness), auditory (loss of hearing, deafness), physical (restricted movement), speech impairment, learning and cognitive difficulties, and photosensitivity. It is made up of four general principles and twelve guidelines, as follows below [22].

Principle 1: Perceivable
The user interface"s information and elements must be presented so that users can perceive them. There are four guidelines under this principle, related to alternative text, time-based media, distinguishability, and adaptability [22].

Principle 2: Operable
It must be possible to operate the user interface and the navigation sections. There are four guidelines under this principle, related to keyboard accessibility, allowing sufficient time, navigability, and seizures [22].

Principle 3: Understandable
The user interface"s content and operation must be comprehensible. Here, there are three guidelines related to readability, predictability, and assistance with input [22].

Principle 4: Robust
The content must be sufficiently robust to allow a wide range of user applications, which use assistive technology, to interpret it reliably. This principle has one guideline, related to compatibility [22]. All the guidelines incorporate success criteria, which comprise levels of conformity relating to the effect of accessibility. To fit the needs of varying situations and diverse users, the guidelines have three levels of conformity [22]: Websites implementing WCAG 2.0 normally apply the 'AA level'. Website accessibility problems can be detected using a combination of automatic tools and criteria set by experts in web accessibility. That said, this process can consume long time and be subjective, as a website"s quality is not necessarily apparent to the expert. Software tools are capable of detecting HTML code and CSS structure errors, assessing browser compatibility, checking links and website performance, and generating warnings [23].

B. The Research Questions
As mentioned earlier, it appears that no study has inspected the accessibility of the hospital websites in Saudi Arabia. Given that it is extremely important that websites provided by healthcare institutions are fully accessible for all, and that there are more than half a million Saudi citizens with some form of disability [6], it seemed appropriate to evaluate the level of accessibility of websites provided by Saudi healthcare institutions. The evaluation covered the highest ranked Saudi hospitals, as of 2020, and attempted to answer the following three research questions.
Research Question 1 (RQ1): How accessible are the websites of top-ranked public and private hospitals in Saudi Arabia?
Research Question 2 (RQ2): Do the websites of top-ranked public and private hospitals in Saudi Arabia differ significantly in their accessibility compliance?
Research Question 3 (RQ3): What are the most common types of errors that affect the accessibility of hospital websites in Saudi Arabia?
The following section explains the evaluation methodology. It includes details of the process of selecting sites and the tool of evaluation employed to rate the accessibility of the websites of the selected hospitals.

III. METHODOLOGY
The evaluation procedure comprised three stages: 1) definition of the websites to be targeted for evaluation, the tool to be used for evaluation, as well as the accessibility www.ijacsa.thesai.org guidelines against which the sites would be tested; 2) evaluation of the targeted websites against WCAG 2.0; and 3) analysis and discussion of the results obtained.

A. Targeted Websites
The 'Ranking Web of World Hospitals', developed by Webometrics, was used as the basis for selecting the targeted websites for the evaluation process. It ranks healthcare institutions around the world on various factors [24]. The sample included the hospitals ranked highest (as of 2020) in Saudi Arabia, and the top ten hospitals from each of the public and private sectors (see Tables I and II). The websites were assessed between September and October 2020. The study specifically evaluated the accessibility of the home page of each hospital"s website. When it comes to accessibility, the home page is considered the most important part of a site. If there are accessibility problems that prevent the home page from being inclusive, then it is very likely that users will encounter access difficulties with the website"s other pages. The home pages of most hospital websites in Saudi Arabia are bilingual, with the content, design and services near-identical in both Arabic and English. Therefore, the researcher decided to evaluate only the Arabic version of the homepages because the Arabic language is the official language in Saudi Arabia.

B. The Evaluation Tool
Out of several tools, AChecker was used for the evaluation process in this study [25]. It was deemed the best evaluation tool for the current study as it is widely used for accessibility assessment and can be applied to individual web pages. It also accepts Arabic websites. AChecker generates a report listing all the accessibility problems found, according to the guidelines selected, in three categories of error: known, likely, and potential. "Known errors" are definite accessibility barriers that are identified. "Likely errors" are issues identified as probably being barriers, but where human input is required for a final assessment. "Potential problems" are issues where AChecker cannot tell if they would have an impact, requiring a human decision [25]. Fig. 1 shows an example of evaluation process in AChecker.
For this study, the criteria in AChecker were set to identify how many errors on each homepage detract from the AA level of conformity in WCAG 2.0. The AA level of WCAG 2.0 guidelines was chosen because, as mentioned earlier, websites implementing WCAG 2.0 mostly apply the 'AA level'. The first step of the second stage of evaluation procedure is to enter the Home page's URL into the tool. The page is then scanned and analysed. The software then displays the results of the analysis, showing the type and numbers of errors and violations encountered. Data collection involved extracting the HTML source code of the web pages of the targeted hospitals (see Fig. 1).
The next section presents and discusses the results obtained.   DISCUSSSION   Fig. 2 illustrates the number of known errors found in all the hospital websites evaluated. A total of 476 known errors were identified in the home pages of the public hospitals (44%), compared to 609 errors discovered in the home pages of the private hospitals (56%), making a grand total of 1085. One of the sites. KFMC had the most amongst the public hospitals. Of the 20 pages evaluated, the mean of the numbers of known errors was 64.7, 47.6 for public hospitals and 60.9 for private ones.  The mean was 514 potential errors across the 20 pages, with 499.2 for the public institutions and 528.9 for the private hospitals. According to the results, there are clearly more potential errors than known errors across the websites examined, while likely errors occur least frequently.       As mentioned earlier, the first research question was formulated to identify the accessibility level of the websites of the top-ranked public and private hospitals' websites in Saudi Arabia. The results obtained clearly show that most hospitals' homepages suffer from accessibility issues. The average number of errors for all homepages was 569.7. This result reveals a serious issue for the attainment of even a minimal conformity with web accessibility guidelines. Only 20% of the evaluated sites has passed the WCAG 2.0 conformance test. It should be noted that the AChecker analysis tool looked for WCAG 2.0 compliance errors at Level AA. If AChecker to be set to search at Level AAA, the expected error numbers would increase by 30% [15].
The second research question was designed to determine if there is a significant difference in the accessibility compliance between the top-ranked public and private hospitals' websites. Table III presents the average number of errors for the websites of the two institution types. Using an independent Mann-Whitney test, no significant difference was found between the websites of the two sectors for any accessibility error type, nor for the total errors. The third research question was set to identify the most common types of errors that affect the accessibility of hospital websites in Saudi Arabia. To answer this question, the AChecker reports were scrutinised. The most common issue by far in hospital sites was related to 'Info and Relationships'. This error concerns content that it would not be possible to present differently without loss of structure or information. Of the websites analysed, 80% have this issue. The second most common error was 'Non-text Content'. Here, non-text content Alternative text is important as it can enable people with visual disabilities to take on board the non-text content"s essence. The third most commonly found error concerned 'Labels or Instructions', which often arises when content requiring user input lacks instructions or labels. The fourth most common issue was that the headings or labels of the targeted websites were not adequately descriptive. The fifth most common issue was 'No Keyboard Access'. Users with mobile disabilities who rely on the keyboard to access a website will be largely affected by this issue.
Overall, the results show that the hospital websites in Saudi Arabia are largely affected by accessibility issues, and to the same extent regardless of whether the institutions are in the private or public sectors. These results are in line with the findings of [12] for Indian hospital websites, which have ongoing accessibility issues and a low rate of compliance with the WCAG 2.0 guidelines. Two specific concerns arise from these findings. Firstly, people with disabilities can encounter difficulties with access to hospital websites. This means they are excluded, whereas people without disabilities are included. Secondly, there is the question of why so few websites follow accessibility standards.

V. CONCLUSION AND FUTURE WORK
The accessibility of any website is a key quality factor. This study aimed to evaluate the accessibility of the websites of the top-ranked public and private hospitals in Saudi Arabia. The results obtained clearly show that most hospitals pay minimal attention to their on-line content"s accessibility. Failure to attain even a minimum degree of conformity to accessibility standards is likely to deprive many disabled people of the benefits provided by a website. Of the websites considered, no significant difference was found in the accessibility compliance between those in the public and private sectors. Furthermore, the findings of the study show that several accessibility errors are being made with respect to the structure of information, non-text content, labels and instructions, headings, and keyboard access. These results can be attributed to a lack of established legislation and guidance related to web accessibility in Saudi Arabia. It is clearly incumbent on the Saudi Government either to set formal web accessibility regulations, appropriate to its national context, or to adapt existing international guidelines and enforce them with legal and other, complementary mechanisms.
It is likely that this study still underestimates the accessibility issues with Saudi hospital websites, as further problems may lie beyond the home pages. Future work could make use of different automated tools and test for more pages. Researchers could also focus on identifying accessibility barriers that appear in the experience of disabled patients in their interaction with hospital websites, rather than simply assessing the compliance of those sites against accessibility guidelines or their ability to pass the scrutiny of evaluation tools. This requires the involvement of end-users to perform comprehensive evaluations for such sites. A further topic for research in the future would be the reasons for so many hospital websites not conforming to the available accessibility guidelines. It would also be worth considering a longitudinal study of hospital websites in Saudi Arabia in order to track progress with web accessibility.

ACKNOWLEDGMENT
The author would like to thank the anonymous reviews for their invaluable comments.