User-Centered Design (UCD) of Time-Critical Weather Alert Application

—Weather alert applications can save precious lives in time-critical risk situations; however, even the most widely used applications may fall short in intuitive interface and content design, possibly due to limitations in the users participation in the design process and in the users range considered. The objective of this study was to investigate whether the application of UCD principles and usability guidelines can improve the use of and satisfaction of time-critical weather alert apps by the public and or expert users. A prototype of a UCD-based weather alert application was developed and evaluated. Initially, thirty-two voluntaries participated in the identification of the important features that lead to the development of the porotype, and then the prototype was tested with another eighty participants (40 young and 40 elderly). The prototype includes five enhancements: auto-suggested location search, an all-inclusive interface for weather forecasts, message alert, visual and intuitive map settings, and minimalism-oriented alert settings. The enhanced functionality was compared to similar functionality in existing commercial weather applications. Effectiveness (completion rate, error count, error severity, and error cause), efficiency (time to completion), and satisfaction (post-task and post-test surveys) were measured. The results showed the enhancements significantly improved performance and satisfaction across both age groups compared to equivalent functionality in the existing app. The Mann-Whitney U test showed a statistically significant difference (p<0.001) in task satisfaction and number of errors between the two apps for all tasks. The Mann-Whitney U test showed a significant difference (p<0.001) in the across all tasks between the two apps Also, overall, young people with existing apps outperformed elderly, and both young and elderly with enhanced apps performed very high. Therefore, the enhancements implemented through the UCD process and usability guidelines significantly improved performance and satisfaction across both age groups to facilitate timely action necessary during a crisis.


I. INTRODUCTION
With the advancement of smartphone technology, countless people have started to receive weather information on the smartphone platform. Due to the requirement for immediate action weather apps are important and potentially life-saving tools in time-critical severe weather events such as flooding and tornadoes. Specifically, usability issues with a poorly designed weather app interface can disproportionately affect certain end-user groups, especially the elderly. Elder users often experience problems with age, such as decreased working memory, and decreased visual, motor abilities, difficulties and cognitive [1][2][3]. In addition to physical and cognitive difficulties, elderly often struggle to cope with the rapid development of smartphone technology [4]; however, smartphone usage among older users continues to grow. As of 2017, nearly 74% of the U.S. population aged 50 to 64 and 42% of the population aged 65 and older owned a smartphone [5]. Numerous studies have focused on age differences in the use of smartphone apps in various fields, including information technology [6], healthcare [7], and communication [8]. However, to our knowledge, there are limited studies on usability testing of weather apps [9,10], and no one has applied user-centered design (UCD) to include a wide range of users, especially elderly.
Drogalis et al. [10] identified usability issues with weather alert applications, such as the inclusion of hidden map menus that required prior counterintuitive actions and the lack of feedback on actions that were performed. Khamaj and Kang [9] investigate usability issues such as poor visualization of critical weather information, inappropriate language usage in time-critical alert messages, and inefficient location search functionality. However, neither of the two studies mentioned above addresses the question of how to effectively meet users' issues and needs. One possible way to address usability issues is to apply a user-centered design (UCD) approach, especially when it comes to end-users of different age groups. In detail, UCD refers to a cycle of design stages in which developers consider the needs, capabilities, and constraints of target users at each stage [11]. Norman [12] and Mao et al. [13] established a general UCD framework and applied to the human factors by Schnall et al. [14]. Careful consideration of each stage is believed to result in an interface that is easy to use and deemed useful [15]. The purpose of this study is to investigate whether the UCD approach could address usability issues for young and elderly when interacting with a smartphone weather alert application.

II. METHOD
The specific contribution of this research is to (1) characterize and classify the issues and needs of users with a broad age range in the design of weather alert applications, and (2) relate these issues and needs to usability guidelines (or principles), (3) design a prototype based on these guidelines, and (4) evaluate the prototype by selecting a target group (i.e., young and elderly). The results could impact the entire design process of a weather alert app (or other similar apps) that could save lives in a time-critical event. Fig. 1 provides a summary of the framework used and applied by the afore-mentioned researchers. Following the framework provided in Fig. 1, the first stage of the UCD process is to describe the user's issues and needs when interacting with an existing application. Issues and requirements can be aligned with major usability guidelines, such as those presented by Nielsen [16] and Rogers et al. [17] and smartphone app design specifications, such as those presented by Gove [18]. At this stage, a broad range of users can be recruited to investigate various problems and needs that they can characterize and classified. Specific to the weather app, questions and needs may relate to any of the features available on any typical weather app, such as location search, weather forecast, alert messages/notifications, alert settings, and radar/maps. The analyzed data can be characterized and classified using content analysis methods. One possible approach is thematic analysis [19]. This approach has been applied to critically evaluate and describe datasets, as well as to implicitly and explicitly identify provided ideas in form of codes refined as themes and subthemes [20,21]. The next stage is to develop a prototype based on the issues and needs of the target end users. For example, if a user asks about the inability to obtain weather forecasts effectively, especially in severe weather, and the need to easily map such forecasts with the locations they add; then these can be design and is implemented by adding location associations to the home screens, which can be accessed with a swipe whenever users open their weather app.
The final stage is to evaluate the developed prototype. In particular, we can con-sider the effectiveness of the prototype by targeting specific populations such as the older. In this case, we need to consider at least two factors: the type of application (i.e. existing vs. new prototypes) and the age group (i.e. young vs. older). Additionally, usability testing can be conducted using several qualitative and quantitative measures, such as completion rates, error counts, error severity ratings, error reasons, completion times, and satisfaction surveys.
III. UCD PHASE 1: UNDERSTAND USER NEEDS AND LIMITATIONS Participants: In this study, 32 participants were recruited voluntarily, were (32 ± 6.4) years (mean ± SD), and ranged in age from 18 to 47 years. Participants were students at the University of Oklahoma and regular users of 11 popular smartphone weather apps running on different operating systems.
Design: The participants were guided by skilled moderators and assistant moderators. Several topics related to user perceptions and behaviors of smartphone weather application usability are discussed. These questions are intended to capture general information: (for example, trends in downloading specific weather apps, prioritization of weather apps in critical and non-critical time weather conditions, and positive and negative usability understandings of weather apps), and specific information: (for example, using different location search methods, controlling alert settings, content and display of critical alert messages, and using menu icons and labels).
Apparatus: A Nikon D3200 camera was used to record the sessions with the participants. A desktop computer, a projector device, and a large whiteboard surface were used to display the questions to the participants.
Procedure: Participants were held in a well maintained controlled environment at the University of Oklahoma. Participants signed a consent form upon arrival and then described the purpose of the study. After that, discussions begin, with each session lasting approximately 90 minutes.
Data analysis: Data was collected through video recording sessions. Afterwards, the collected data was transcribed, characterized and classified using qualitative thematic analysis [19]. Specifically, using thematic analysis methods, the data are: 1) transcribed verbatim, 2) encoded using representative words and phrases, and 3) re-fined into common themes.

A. Results of Phase 1
Participants presented three main themes: usage efficiency, user cognitive load, and effectiveness. These themes are primarily associated with common features on most popular weather apps (location search, weather forecast, alert messages, alert settings, and radar maps), as detailed below.
1) Usage efficiency: Participants were highly concerned with the time and number of steps required to access timecritical information, such as in a weather app. Participants shared several examples of inefficiencies in the design of the location search feature in popular weather apps. For example, in addition to conventional search methods (such as zip and/or city), the widely used weather app Weather Radio has recently adopted a precise location search feature (which enables users to search and save the specific locations) to provide users with accurate weather forecast. The application restricts users from searching for the different locations as required to navigate on the map and identify the desired location (see the example in Fig. 2(a)). Therefore, end users need an effective search method, since this feature needs a great degree of knowledge of the geographic area of the map, widespread visual attention, www.ijacsa.thesai.org and repeated zooming in/out inside the small smartphone screen. Another efficiency requirement has to do with weather forecasting (such as humidity and temperature) feature. Users indicated that a limited number of steps were required because they felt it was accessed more frequently than other features. Several current weather applications involve multiple steps on several screens to access weather forecasts (see examples in Fig. 3(a-d)). 2) User cognitive load: Due to critical situations associated with extreme weather conditions, users also expressed a need for weather apps to simplify their cognitive processes and decision-making strategies. For example, users place high emphasis on the requirement for concise and structured push alert messages throughout severe weather conditions; alerts are created by the weather agency's systems and automatically directed to third parties, including weather apps, where they push them to the end user exactly as they are received. This is because alerts often contain technical data (such as geographic area codes) and cluttered data (see the example in Fig. 4(a)), which can hinder users' understanding of the alert and rise their cognitive load. Additionally, alerts often contain a huge quantity of evidence; most of this is not related to user-saved locations (e.g. names and information for all alert areas). While retrieving information about distant or isolated locations is critical for some operators in certain usage contexts, such as when traveling, making them a major part of alerts can be detrimental to users. Users also need a radar map feature that is built-in to identify and understand. Participants shared several examples of few popular weather applications that lack intuitive and/or visible instructions on in what way to access maps of the respective saved localities or control their settings (see examples in Fig. 5(a, b)).
3) Effectiveness: Owing to the limited smartphones screen size and the critical information displayed, users demand optimized and flexible functionality. For example, in the alert settings feature, the weather app either lets users control of all types of weather alerts and sub-alerts (see example in Fig.  6(a, b)), or does not permit the control of any alerts, but automatically uses active warnings as alerts. For this reason, users have expressed concern about large numbers of weather warnings, most of which are rarely needed and / or insignificant for the average user. Additionally, notifications that push any active alerts without end-user control are considered mandatory interactions. Therefore, users have expressed the need for the ability to control only a few relevant alerts.

IV. UCD PHASE 2: DEVELOP PRODUCT PROTOTYPE
A prototype weather app was developed after understanding the specific concerns and needs of users and taking into account widely used smartphone design heuristics, age-related constraints and needs, and common usability guidelines. The prototype application was designed using the InVision application software (https://www.invisionapp.com/company) called "EZ Weather".

A. Design Proposal Overview
The proposed design solution for the weather app features has been implemented in the EZ Weather app. To address the location search problem, a similar approach to the Google Maps app could greatly improve the efficiency of this feature: enter a familiar location with effective auto-suggestions. With this feature, the user can add an exact location by typing the location name/address (e.g. hospital name/address) in the search bar of the application and selecting it from the autosuggest list (see example in Fig 2(b)). To reduce the time and number of steps to access weather forecasts, it may be helpful to have all weather forecasts for each saved location in one screen: all-inclusive weather forecast in one screen (see example in Fig 3(e)). For alert messages, a method for filtering the content of the message can significantly reduce the cognitive load on the user by including only relevant and necessary information on the main alert screen that is relevant to the user's saved location; all other information can be accessed from the secondary menu, including distant underalert area.
In addition, simple language and hierarchically structured information using everyday language can enhance user understanding of messages and responses to alert threats. See Fig. 4(b)), for an example, content that is structured, prioritized, and language simplified. In addition, to facilitate the user's identification and understanding of the radar map feature, it may be helpful to display the list of all saved locations; from which the user can visually select the desired location; then the map of the selected location and its settings are displayed: use the visible and intuitive Map menu (see example in Fig. 5 (c & d)).
To avoid an excessive number of alerts that the user needs to control and interact with, and to give the user the freedom to control the push alert notifications, it may be beneficial to include the most critical and common alerts (on/off) to control. In bad weather, users will receive alert notifications about the time-critical alerts they have turned on (e.g. tornado warnings). Alerts that are not time-critical, such as wind, are not spontaneously sent to the user as notifications; these alerts only appear after clicking the respective symbol on the affected location screen (Fig. 6 (c & d)). Other key usability and smartphone application guidelines may also need to be considered to improve the overall user experience. These guidelines include labeling all icons with representative text labels, using short descriptive information next to menu options, providing appropriate feedback on user actions, and using appropriate colors with easily distinguishable contrast to visualize and indicate weather conditions (e.g. red screen for locations under a warning alert), provide shortcuts (for example, constant main icons across all screens), text in relatively large fonts, and use a consistent and intuitive layout. Apparatus: For the present study, both the applications i.e., Weather Radio (version 3.0.5) and EZ Weather (version 1.0.0) are installed on iOS Smartphone. Using high fidelity simulations (InVision), a powerful interaction design system, we displayed recorded alert messages on the Weather radio at any time during the experiment. To record user interactions, in particular to calculate the time to complete a given task, to www.ijacsa.thesai.org count and classify errors the Nikon L340 camera was used. Demographics, post-task and post-test surveys were printed on paper.
Scenario & Tasks: Participants were given a scenario in which their grandmother was admitted to Mount Auburn Hospital in Cambridge, MA, and noticed that a flood warning had been announced in the Cambridge area. To understand grandmother's exact location alert risk level and access all relevant information, participants can search for and add to grandmother's exact location, access and identify relevant weather forecasts and alert messages. Further, participants need to view alerts on map in map settings and adjust alert settings to receive relevant alert notifications. Fig. 7 shows the successful completion of the scenarios and tasks.
Procedure: First, participants were explained the purpose of the survey, signed a written consent form, and then completed the demographic survey. None of the participants received training and were not given the opportunity to practice themselves using the test app; the participants were given scenario and task guidelines and were asked to start the experiment after they were told they were ready. Weather Radio is a running application, so participants won't see any alert messages unless during an active alert, so Weather Radio's alert message assignment was presented to participants through a smartphone interface designed by InVision. The alert message displayed to participants is a flood warning posted to Weather app users in Cambridge, MA on April 1, 2017. After finishing each task, participants were asked to complete a posttask survey to provide their instant feelings about each task/ feature. Finally, participants were asked to complete a post-test survey to evaluate their satisfaction with the test app.
Experimental Design & Variables: Full factorial design (2 levels of app * 2 levels of age group) were used in the present study as independent variables. The dependent variables were effectiveness, efficiency, and user satisfaction. The control variable was user experience; all users in both age groups must have at least six months of experience with iOS smartphones and the Weather app (excluding the app tested). Error metrics are the reasons for errors made (usability issues), how often they occur, the proportion of users who make mistakes, and the severity level of the cause of the error. The error severity ratings used in this study are based on a rating scale proposed by Nielsen [23] (Table I).
Finally, Pearson correlation test (r) was also performed to define the relationship between the usability measures used in this study.
The post-task survey was done by using a Single Ease Question (SEQ) with 7-point Likert rating scale [22]. This question was used because it was found to be as effective as other complex measures of task difficulty, namely the Usability Magnitude Estimate (UME) and Subjective Mental Effort (SMEQ) questionnaires [23]. The post-test survey was done by using User Interface Satisfaction (QUIS) questionnaire [24]. Fig. 7. (a-e). Process of performing tasks on Weather Radio vs. EZ Weather Data Analysis: All data's were analyzed using SPSS version 23. A two-way analysis of variance (ANOVA) between subjects was performed to determine the effect of both application used and age group task completion time for all tasks. The Mann-Whitney U test was used to examine the size of the differences between the app and age group variables used, both in terms of the number of errors and the metrics of the post-task satisfaction survey. For post-test satisfaction surveys, mean and standard deviation errors were calculated.  Fig. 8 shows that all users in both groups were able to complete a given task in EZ Weather successfully. However, several users were unable to complete all three tasks on Weather Radio, and older users had a higher failure rate on the location search task.

Alert Settings
Users do not understand the function of home screen icons because these icons are neither marked nor standardized in smartphone applications. Also, the large number of alerts and sub-alerts seems to confuse users about the options required for the task. 186 256 3.5 b) Errors: Fig. 9(a-e) shows that errors on EZ Weather are significantly less than those on Weather Radio for all tasks, for both young and older users. The results showed that the older users had made more mistakes on both the apps compared to younger participants on all tasks except Weather Radio's alert message task and EZ Weather's location search task. The Mann-Whitney U test showed a significant difference (p<0.001) in the number of errors across all tasks between the two apps (Table II). The results showed no significant error differences for all tasks between the both groups, except for the weather forecast and map setting tasks. For other error-related metrics, almost all users in both groups made errors, and with different frequencies, due to the usability issues with every task in Weather Radio except the Alert Messages task (Table III). Nearly half of the participants made mistakes due to alert message usability issues. In contrast, fewer users made mistakes in EZ Weather's tasks, with a much lower frequency and average severity rating, compared to Weather Radio. Errors on EZ Weather are mainly caused by swipe actions (such as spelling mistakes and accidental clicks on adjacent function icons) (Table IV).

Frequency of issue
Average Severity Rating

Young Users
Older Users

Location Search
Users made typing errors when typing location name. 17 9 0

Weather Forecast
Users couldn't easily figure out that accessing weather forecasts of different locations was through swiping the screen right or left. 10 12 1.5

Alert Message
Users didn't expect the alert message icon to be clickable and/or required to access the message, when clicked.

Map Settings
Users mistakenly tapped adjacent icons of unrelated functions. 4 6 0

Alert Settings
Users mistakenly tapped adjacent icons of unrelated functions. 2) Efficiency (task time): Fig. 10(a-e) shows that, on an average, the younger and older participants take fewer time to accomplish each task of EZ Weather compared to Weather Radio. The two age groups had relatively taken same times on all tasks for both apps. Therefore, two-way ANOVA (Table  V) shows that on both weather apps, there were no significant time differences between young and elder users for all tasks except the map setting task. In the weather app tested, however, the time to complete all tasks was significantly different (p<0.001) between the both groups. In addition, there was no significant interaction between the both group and the app used for all tasks.
3) Post-task satisfaction ratings: Fig. 11(a-e) shows that both young and older participants found the EZ Weather task easier than the Weather Radio task. Younger users rated the ease of use for both apps higher compared to older participants, though the difference was not huge. In addition, the Mann-Whitney U test showed a statistically significant difference (p<0.001) in task satisfaction between the two apps for all tasks (Table VI). It also showed that young and older users were similarly satisfied with all tasks.

4) Post-test satisfaction ratings:
Overall, the results in Fig. 12 and 13 was based on the QUIS questions [24], where the same questions were asked to both groups of participants under same parameters, suggesting that younger and older participants feel comparatively similar to the interface de-sign specifications, and younger users are more satisfied. In addition, all participants were very pleased with EZ Weather, while most users of Weather Radio were disappointed. Users report that the screen design of EZ Weather is very good, the text and images are very clear, the fonts are very clear, the amount of information presented is sufficient, and the arrangement is reasonable. However, participants were appeared to be disappointed with the amount of information displayed on Weather Radio; they observed that there was too much information, particularly about time-critical alerts.
Users have found consistent terminology and messaging throughout EZ Weather as well as the location of on-screen instructions. Participants reported that performing any action produced predictable results in EZ Weather, with a very acceptable delay between actions. However, many participants noted that they were not sure what to expect when performing multiple actions in Weather Radio. Compared to Weather Radio, users report that using EZ Weather is very easy to pick up, learn, and efficient. Additionally, users seem to be very pleased with the number and order of steps required to accomplish each EZ Weather task. They also liked the feedback they received as they completed each task. Navigating features and remembering names and commands was also easy for participants in both groups. When it comes to the EZ Weather feature, although many users made no mistakes during the experiment, they reported that errors and typos were easy to correct. Users also like their ability to perform or undo actions using shortcuts. In addition, they say that users of any experience level can easily and consistently complete their tasks. The choices of color for both apps seem natural enough for most users, with EZ Weather having a higher user satisfaction rate. Tables VII and  VIII show the Spearman rho correlation (ρ) results among all the indicators in this study. Since all users successfully performed a given task, the correlation between completion rates for the two age groups and other metrics on EZ Weather was not performed. The results showed a strong positive correlation between task time and errors, and between posttask and post-test satisfaction, for both age groups for both applications. However, the time spent on each task and errors were negatively correlated with post-task and post-test satisfaction scores.

VI. DISCUSSION
Overall, this study establish that the UCD weather app (EZ Weather) was considerably more useful compared to the other popular weather app (Weather Radio) for all the evaluation metrics used. Also, younger and older participants seem to have comparable results on all tasks for both the weather application, with younger users experiencing marginally greater performance and satisfaction. These results show that regard-less of age, an application's interface design significantly affects end-user performance and perceptions (positive or negative) of application usability. The results further demonstrate that adopting a UCD approach to applications that contain time-critical data, such as weather applications, will result in extremely interactive and usable systems. Results for all metrics had showed that prioritizing and structuring critical evidence and using everyday terms throughout the EZ Weather interface greatly helps participants easily interacting with inherent features and finds them beneficial. For example, all young and older participants were able to complete alert message tasks in EZ Weather with far greater efficacy and task satisfaction compared to Weather Radio features.
Additionally, only a handful of users had errors in some of the usability issues category in EZ Weather's alert messages task, while nearly half of participants of each group had errors in this task for Weather Radio due to usability issues in the catastrophe category. In addition, the alert message task further demonstrate that the push alerts will greatly help end participants to correctly perceive alert threats and respond effectively to alert threats, particularly in the forthcoming weather conditions. The greater usability of the EZ Weather is also due in part to consideration of efficiency principles during the design phase. This is supported by the very limited time and steps required to access any feature on EZ Weather. A good example is the weather forecast feature that includes all weather prediction information for respectively saved location within same screen (all-inclusive weather forecast); with just one swipe right or left, the user can access other saved locations' weather forecast. Although some users couldn't figure out the swipe functionality, from the first test, all success-fully completed the weather forecast task with significantly greater efficacy and task satisfaction, with fewer and less errors, than Weather Radio multiple-screen weather forecasts feature. Another example of an efficient feature of EZ Weather is the location search feature. With efficient auto-suggestions to type in familiar locations, both groups of users can complete the corresponding task in seconds. By contrast, navigating maps and pinpointing locations within the limited smartphone screen size is not only inefficient, but also ineffective, especially for older users, as demonstrated by Weather Radio's high failure rate and completion time for location searches.
Users' greater performance and satisfaction with EZ Weather than Weather Radio may also be related to EZ Weather's simple design. For example, limiting user-controlled alerts to the most common and critical alarms on EZ Weather (using a minimalist alarm setup) enables users to effectively and efficiently perform alarm set-up tasks with great satisfaction. Moreover, EZ Weather also permits participants to use non-critical alerts (via an illustrative icon displayed on the exaggerated location screen) during active alerts. The significant drop in performance and satisfaction with Weather Radio's alerts setting tasks was primarily because of the huge number of alerts and sub-alerts that users need to navigate and control (e.g., 16 sub-alerts for wind alerts alone). Two other important principles considered in the design of the EZ Weather interface are its usability and ease of recognition. For example, while the map setting task on both apps requires interaction with similar step and function icons. EZ Weather's greater performance and satisfaction on this task is supposed to be recognized to the visualization and intuitive menus (using the visualization and intuitive map menu), and appropriately labeled icons. These factors and results show that the actions have been performed with expectable results in a logical sequence. However, invisible elements in smartphone apps (such as the map settings menu bar) and icons that are neither marked nor standardized may be one of the main reasons for the dramatic drop in performance and satisfaction with Weather Radio tasks.
The usability and design principles monitored in the design of EZ Weather that may have ultimately contributed to great results include: feedback on system status (e.g. confirmation messages to perform activities), uniformity of application elements (e.g. settings menu), shortcuts usability to expedite interaction and correct errors, provide short descriptive information to help users understand the functionality of the corresponding feature, and use large text fonts along with high contrast and indicative colors to address age-related limitations for older users. The post-test satisfaction survey results were consistent with those of the task-based metrics. The two apps differed significantly in satisfaction with all interface standards across the two age groups, suggesting that applying usability guidelines to interface design not only results in high performance but also satisfaction for users. Also, on most QUIS survey items, the two age groups are highly similar in satisfaction, meaning that age differences do not significantly affect user satisfaction. The strong correlations between all used metrics complement the results of previous study [25], which suggest that usability metrics are interdependent aspects that inform the usability of an interface. However, Frøkjaer et al. [26] showed the dependence of usability metrics depends on whether the tested interface contains highly complex functionality. If the domain of interest includes complex features, a weak correlation is expected; if not, a strong correlation is most likely.

VII. CONCLUSION
This study validated the usability of the UCD weather app (EZ Weather) by comparing it to a popular app (Weather Radio). In addition, this study also considered young and older users to determine whether the usability of the test app would be highly influenced by age differences. Results for all metrics showed that users' performance and satisfaction on EZ Weather improved significantly regardless of age, demonstrating the importance of considering UCD methods, usability heuristics, and smartphone design rules in interface development.