Student Information System: Investigating User Experience (UX)

There is growing recognition that electronic student information systems support college administrations and enhance student performance. These systems must fulfill their user’s needs (efficiently achieve their academic goals) while also providing a positive user experience (UX). This study used quantitative and qualitative approaches to elucidate students’ perceptions and investigate UX toward the SIS currently used at the Public Authority for Applied Education and Training (PAAET), a higher education institution in Kuwait. Survey data collected from 645 PAAET students were analyzed to determine their perceptions of and experiences using this SIS. The findings revealed that students had a slightly positive UX with this SIS. The system’s perspicuity, stimulation, and dependability were rated slightly higher than its novelty, attractiveness, and efficiency. The most pertinent usability issues that focus on the human interaction with systems were identified and discussed, hoping that it will allow officials and SIS system developers alike to make relevant and impactful improvements to newer versions of these systems. These results shed light on the need for continuous SIS evaluation and a broad research scope to develop innovative SIS with intelligent functions for novel activities. Such features enhance students’ interactivity and productivity, which encourage their academic success. Keywords—Student information system; user experience; usability; human-computer interaction; e-learning


I. INTRODUCTION
Successful student information systems (SIS) make students productive and improve the workflow of their academic services [1]. These systems, including learning management systems (LMS), provide functions and tools that overcome college-level administrative and academic problems [2,3]. SIS allows college students to manage their data, including registering in courses, maintaining grades, showing transcripts, and generating progress reports. Although SIS are widely used in the academic world, these systems require regular evaluation to make them more productive. Having effective and efficient SIS significantly impacts stakeholder groups' operation and performance [4,5]. Therefore, the key features of SIS must be identified, and appropriate evaluation criteria must be developed to measure them.
Usability is associated with the user acceptability of any system [6]. Determining the usability aspects is essential because millions of people, including students, instructors, and administrative staff, use SIS to conduct administrative and academic tasks. Recently, user experience (UX) has gained considerable attention among researchers in academia and industry and has become a vital aspect of the products' success [7]. The author in [8] stated that UX is considered a key aspect in designing products and services. It is argued that institutions that apply UX design activities in their system development achieve many potential advantages that increase user satisfaction. The author in [9] believes that an effective UX does not appear on its own but must be systematically evaluated. Due to its importance, several frameworks and models have been proposed to design and assess the UX of interactive systems. These models serve as a guide to improve the design and the quality of interactive systems [10].
Although the literature provides UX evaluations with various information systems, it does not do so for SISs [11]. Designing usable SIS is essential; however, little research was conducted, especially in universities among Arab Gulf countries. Several usability studies did not analyze and develop such systems considering students' perceptions and their UX. This observation led to the work presented in this paper, which tries to fill this gap by investigating student experience with SIS. This study was conducted to elucidate students' perceptions of UX with the SIS used at the Public Authority for Applied Education and Training (PAAET). It is a pioneer study given the absence of research on this topic and the context of Kuwait's educational system [12]. Its significance is to provide system developers with pertinent improvement possibilities for future versions of this SIS to enhance efficiency and attractiveness and improve users' interactions with the system and its related functions [13].
This article is organized into sections. Section 2 reviews the relevant literature; Section 3 explains the methodology. The results and a discussion thereof are presented in Section 4, and Section 5 draws conclusions and explores future directions.

A. Evaluation of SIS
One of the critical systems for managing HE's administrative and academic aspects the SIS [14]. Although SIS are widely used in the academic world, these systems require constant evaluation to ensure their relevance and effectiveness [15,11]. An effective SIS not only satisfies administrators and students but also ensures sustainable academic progress [1]. Determining the usability level of an SIS from the human-computer interaction perspective is an essential consideration for universities. Developers, therefore, need to continually be creating better, more usable systems www.ijacsa.thesai.org informed by understanding their potential users with concerns of social and cultural issues [16]; individual differences [17]; and gender and age differences [18].
Some research focuses on SIS development, while others investigate SIS usability, UX, and perceptions. The author in [19] described the design and development of a novel SIS. The study was motivated by the fact that there are difficulties associated with the manual methods used to manage student information at the University of Diyala and aimed at adopting new SIS to increase efficiency and accuracy that also helps college administrations speed up the decision-making process. Besides, [20] developed an SIS for the Faculty of Electronics & Computer Engineering, University Teknikal Malaysia Melaka. They described the development steps needed to operate the system. Their system focused on recording and updating students' records system replacing the old traditional SIS. They believed that this system would contribute to new knowledge in the field, ease use, and better arrangement and scheduling.
Considering the usability of SIS, [21] conducted a study to examine the usability factors of an SIS at a public university. Data were collected from 132 computer science students using a questionnaire. The authors used factor analysis, which involves the user's perceptions of usefulness, speed, interface, and error corrections. The results demonstrate that several usability attributes, such as the importance of information and system functionalities that are commonly gathered, affect user engagement. A similar study was carried out by [22] to examine an SIS's usability at Near East University. The results provide recommendations to improve the interface and enhance system attractiveness. In addition, an empirical study conducted by [11] elucidated the influence of student background on SIS experiences in terms of emotion, performance, and perceived usability. Substantial variations between user emotion, performance, and perceived usability were found.
The author in [14] states that SIS is critical when managing the administrative and academic aspects of HE. Their study investigated how system quality, information quality, and information presentation impact academic and administrative staff satisfaction. Data collected from 120 users were evaluated using factor analysis and regression, revealing that system quality and information quality have significant indirect effects on user satisfaction while information delivery does not directly or indirectly. The author in [23] evaluated SIS performance toward improving the current SIS productivity at Kalinga State University using an interviewbased approach to elucidate students, administrators, and instructors' perceptions. The system was found to satisfy five usability factors: usefulness, functionality, reusability, maintainability, and security.

B. User Experience (UX)
Usability and UX are often confused. The author in [24] believes that usability is mainly concerned with the functional part of a system. At the same time, UX is related to how the users interact with a system that involves the user's feelings and attitudes. Similarly, [13] claimed that UX focuses on understanding users, their needs, interests, strengths, and limitations. She stresses that investigating UX helps to improve users' interactivity with the system and raises their perceptions. Similarly, Norman and Nielsen stated that UX involves the users' perception of usability, which examines how users view the usefulness and effectiveness of the system or application [8,25,26].
The author in [7] developed a User Experience Questionnaire (UEQ) that measures UX. The six scales of this questionnaire comprehensively represent UX by quantifying six dimensions of usability. These dimensions are attractiveness -the product should look attractive, enjoyable, friendly, and pleasant‖; efficiency -the user should perform tasks with the product fast, efficient and in a pragmatic way‖; perspicuity -the product should be easy to understand, clear, simple, and easy to learn‖; dependability -the interaction with the product should be predictable, secure and meets my expectations‖; stimulation -using the product should be interesting, exciting and motivating‖; novelty -the product should be innovative, inventive and creatively designed‖ [27]. The author in [28] investigated the impact of culture on the UX of a system using the UEQ. They examined how Indonesian and German students evaluate common systems according to their UX and provided insights and possible explanations for any detected cultural differences. Other studies that have used the UEQ to evaluate UX include [7,29,30].

III. METHODOLOGY
The research methodology is described in this section, including the research sample, instruments, and procedure.

A. Research Sample
This study included 645 participants from the five PAAET colleges: College of Basic Education, College of Business Studies, College of Technological Studies, College of Health Sciences, and College of Nursing. Table I presents the demographic data and sample distribution of the study population (gender and college).

B. Research Instruments
This study used both quantitative and qualitative approaches to assess the UX of PAAET's SIS. A focus group was conducted before administering the online questionnaire to the entire study population.

1) Focus group:
A focus group was administered to gain confidence in a tentative UX questionnaire that is to be used as a tool to evaluate the UX of the SIS. A single face-to-face focus group session was administered by a facilitator at the www.ijacsa.thesai.org College of Business Studies (CBS), one of PAAET's colleges, during the fall term of the 2019-2020 academic year. Sixteen CBS students with over 30 credits were chosen randomly to participate in the focus group session to validate the questionnaire's statements. The questionnaire's statements were discussed and verified in terms of content for validity by the 16 participants. Statements that seemed ambiguous or redundant were highlighted. The resulting in-depth discussion within the focus group provided further insight into the students' perceptions of the on-line system in question. The findings were documented in a report approved by the participants five days after the focus group discussion.
2) Questionnaire: The questionnaire used for this study was adapted from the UEQ designed to measure UX that can be found at www.ueq-online.org. The author in [7] reported that the UEQ reliably depicts six UX dimensions, including attractiveness, efficiency, perspicuity, dependability, stimulation, and novelty. For this study, the authors chose to use the six dimensions differently than described in the handbook while still following its recommendations to "use terms that fit the language of your stakeholders" [7]. The items were modified to reflect the specific nature of PAAET students. The rationale was explained in short sentences, rather than singular words, to avoid students questioning the meaning of the phrase, which is the expected behavior of PAAET students, confirmed by the focus group. Fifty students piloted the adapted UX questionnaire during the fall semester of the 2019-2020 academic year while the focus group was administered. The objective was to find any ambiguity in the statements and alter them accordingly. The final version of the questionnaire, consisting of 23 statements mapped onto six scales, was shaped by the focus group's findings and the pilot study.
The final questionnaire used in this study consists of seven parts. Part 1 collects students' demographic information (gender and college). Parts 2 to 7 evaluate the UX of the SIS system and assess the six usability dimensions rated using a five-point Likert-type scale (1 = Strongly Disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, and 5 = Strongly Agree). The questionnaire was developed to measure students' perceptions of PAAET's SIS. A pilot study was conducted to test the validity and reliability of the questionnaire. The questionnaire's internal consistency was confirmed by determining the correlations between each theme and the questionnaire's total score, obtained from surveying 50 students. The researchers used SPSS to calculate the correlation coefficients in Table II. The correlations between the individual dimensions and the overall score were high (p < 0.01) and ranged from 0.795 to 0.901, indicating high internal reliability and construction integrity.
Similarly, the questionnaire's reliability was established by calculating the Cronbach's alpha for each dimension using SPSS (Table III). The questionnaire's dimensions exhibited a high degree of reliability, with coefficients that ranged from 0.74 to 0.93. The total Cronbach's alpha score was 0.96; therefore, the questionnaire was reliable and generalizable.

3) Research procedures:
The researchers developed the quantitative UX questionnaire and conducted the qualitative focus group discussion. The focus group session was organized and run by a facilitator who began by stating the study's objective and emphasized the students' feedback on improving the SIS. Informed consent was obtained, and the students were assured that the information extracted from their feedback would be used for scientific research only. The facilitator instructed the participants to introduce themselves, some of whom knew each other, which seemed to improve the group dynamics. The focus group discussion was guided by the sequence of the statements in the UEQ. Each group of statements within a construct was carefully read out loud by different participants. The data collection relied on the facilitator's notetaking during team discussions and the participants' written comments. The session lasted for about 50 minutes.
The focus group feedback was analyzed using the -three coding-framework‖ reported by [31] . This helped to understand the current state, problems, and opportunities for the system and help shape the questionnaire's final statements. The focus group students simultaneously found the statements in the questionnaire clear, easy to follow, and doable. Nevertheless, they all have agreed on the need to merge/delete some statements, as shown in Table IV.
The focus group session helped to shape the final version of the questionnaire. The questionnaire was administered online during the fall term of the 2019-2020 academic year. After obtaining approval from PAAET's higher administration, the questionnaire was distributed to all faculty, who, in turn, were instructed to forward it to their students. Responses were collected over seven days. The results were analyzed using SPSS and frequency, percentage, mean, standard deviation (SD) was used in the analysis.

IV. RESULTS AND DISCUSSIONS
A. Students' Perceptions of the SIS This section presents the results of the analysis of students' perceptions of the SIS. Tables V to X include 23 items distributed in the six dimensions: attractiveness; efficiency; perspicuity; dependability; stimulation; and novelty. The tables presented in the following sub-sections show the percentages, means, standard deviations (SD), and ranks of the items within the dimensions according to their mean values.

1) Attractiveness:
Attractiveness refers to whether the system looks appealing and pleasant to the user. Table V lists the three items used to investigate the attractiveness dimension. The mean values of items A1 were the highest, which indicated that the system screen was exciting (mean = 3.29). Item A2 that the SIS was interesting, came in second (mean = 3.20), while A3 "The SIS interface is attractive" scored the lowest in this dimension (mean = 2.95). Visual design is a non-functional element designing interfaces and confers attractiveness to any given system [32,33]. The analysis of students' responses summarized in Table V, revealed that the SIS' attractiveness was marginally appreciated. The attractiveness dimension was ranked fifth of the six dimensions as the mean value was 3.14, slightly higher than the neutral point 3.0.
Aesthetics is a set of principles that relate to a design's attractiveness. The visual design includes consistency, color, association, pattern, scale, outline, and visual weight. It engages users by helping them to perform the correct functionality on the system smoothly [34]. System designers should use aesthetics to enhance their designs' usability, innovation, and attractiveness [35,33]. Visual design is a crucial success factor; however, its importance has changed over time. The author in [36] investigate the dynamics between the significance and the attractiveness dimensions of software product features and their influence on user satisfaction. The study provided useful insight into the tradeoffs between the attractiveness and importance dimensions and informs which features should be focused on evolving software products.
2) Efficiency: Three questions were used to examine the efficiency of the SIS (Table VI). Efficiency implies that users can perform their tasks quickly and without unnecessary effort. As for students' responses, item E2 -I believe that the SIS meets my requirements‖ was ranked highest with a mean value of 3.27, which indicates that the SIS is marginally efficient and meets students' needs. Item E3 also showed that the SIS was ranked slightly effective (mean = 3.26). However, item E1 -All system commands are executed quickly‖ was ranked lowest with a mean value of 2.78, which was below students' expectations. Efficiency is essential to usability, which measures how quickly users can accomplish their tasks and, as such, positively impacts system quality [37]. The efficiency dimension was ranked lowest with an overall mean value of 3.10. This indicates that students are neutral to agree that the SIS is efficient slightly.
3) Perspicuity: Perspicuity refers to the simplicity of the system, easy to use, and easy to learn. Table VII shows five items used to investigate perspicuity. Item P2 -It is necessary to have a clear explanation of how to use the SIS‖ ranked highest with a mean value of 4.07. In contrast, item P1 indicated that students were neutral concerning whether they received enough training to use the SIS and ranked the lowest (mean = 3.18). Perspicuity considers how easy it is for users to learn to perform a task using the interface and how easy it is to remember how to perform it. This dimension was ranked the highest of the dimensions with an overall mean value of 3.58, which indicated that students moderately agree that the system is easy to use and learn. Learnability is measured by the level of ease with which users become proficient at using a system [38]. The author in [39] stated that learnability is one of the five quality dimensions of usability; the others are efficiency, memorability, satisfaction, and error. Students moderately agreed that the SIS was easy to use, systems commands were understood, and the system can be used without the help of others. During the focus group session, a few students suggested conducting training and orientation sessions on using the system's functions. Others said that they did not use all the system functions, focusing on basic functions that allow registrations and viewing their grades and schedules. According to [40], the learnability and user-friendliness of a system are inversely proportional to the amount of training time needed for its use. Focusing on the design helps to increase learnability and ease of use by allowing users to understand the interface quickly without training. Besides, consistency in interface design makes the system's menus and commands well organized and easy to use; inconsistencies can confuse systems.
Training and guidance are critical issues for the proper use of technology in educational institutions. It is stressed by [41,42], that colleges and universities should provide adequate training and guidance for students and advisors to use and utilize the systems' tools and functions.

4) Dependability:
Dependability refers to whether the user feels in control of the system and the interaction with the system is predictable. Table VIII includes four items used to investigate dependability, namely, system reliability, expectancy, accuracy, and security. The students' responses to item D4, -The SIS is secured‖ (mean = 4.02) were the highest. In contrast, item D3 -The SIS meets my expectations,‖ ranked lowest with a mean value of 3.17. The dependability dimension ranked third of the six dimensions with a mean www.ijacsa.thesai.org value of 3.51. This indicated that students moderately agree that the system is trustworthy.
Dependability is a non-functional property of a system derived mainly from whether users can trust the system. An alternative concept that also contributes to dependability is avoiding system failures that are more frequent and severe than acceptable [43]. Dependability encompasses many attributes, such as the system's reliability, availability, durability, accuracy, and security [44]. Software designers should value this dimension highly because dependable software is often praised and recommended by its users. The author in [45] claims that dependability can provide services that can be trusted defensibly within a given timeframe.

5) Stimulation:
Stimulation queries whether the system is exciting, motivating, and fun to use. The four items in Table  IX show that the SIS was moderately stimulating. Item S2, -Displaying the number of courses that I have completed, and the remainder is valuable,‖ ranked first (mean = 4.16). Second, item S3's mean score of 3.57 indicated that the SIS motivated students to do better. Item S4, -The SIS is an interesting system," ranked lowest with a mean value of 3.22. The stimulation dimension was ranked second of the six dimensions with a mean value of 3.55. This indicated that students moderately agreed that the system was stimulating. To achieve a successful design that has a positive impact on the user and achieves a business objective, persuasive elements must be explicitly considered in the context of the behavior that the application seeks to influence; and this must take place in the early stages of the design process. The author in [46] listed motivational drivers for system developers, which are an excellent place to start with any application: -collecting, connecting, achievement, feedback, reciprocity, and blissful productivity.‖ 6) Novelty: Novelty reflects whether a system is innovative and creative. Four items are shown in Table X investigating novelty. From the mean values of each item, the students' responses to item N3 -The SIS is technically advanced‖ ranked highest with a mean value of 3.51. This indicated that students slightly agree that the system is novel. Novelty can catch the user's attention and is defined by [26] as -The quality of being new, original, or unusual.‖ Software novelty can help a system be noticed among the many other systems and applications; however, to do so, the system must also be useful for users. Reference added other aspects of novelty that contribute to UX, such as creation, invention, and innovation.  During the focus group session, some students expressed that while the SIS allowed them to achieve their intended goal, the interface looks traditional and boring. Some said the SIS does not provide intelligent -what if‖ scenarios concerning course requirements and scheduling or suggestions to boost their GPAs. An intelligent expert can provide personalized support to each student and creates a virtual collaborative environment that includes advisors, students, registrars, and IT staff to ensure that the SIS effectively contributes to student success. It is essential to inject creativity into the design of these systems. The author in [47] stresses that creative design can improve the efficiency and utilization of a system and claim that there are benefits to using creative approaches to design and develop innovative new models for software presentation and information retrieval.

B. Comparison of UX Dimensions
The results of the analysis revealed that the students had a marginally positive perception of the SIS. A comparison of the six dimensions of UX is illustrated in Fig. 1. The mean values are used to indicate the level of the six dimensions of UX: perspicuity, mean 3.58; stimulation, mean 3.55; dependability, mean 3.51; novelty, mean 3.29; attractiveness, mean 3.14; and efficiency, mean 3.10.
For the six dimensions, the means ranged from 3.10 to 3.58, with an overall average of 3.36. Three dimensions were above 3.5, which suggests that the SIS was slightly appreciated. With the ongoing evolution of tools and applications, software improvements are essential, and research must best inform the most pressing. Emphasis should be placed on the efficiency, attractiveness, and novelty dimensions of this SIS. The current study investigated the UX of an SIS, a key platform in any contemporary academic institution's operation, by analyzing students' perceptions. The strengths and weaknesses of the design/usability/UX provided by the SIS currently used at PAAET were examined according to six factors that are central to successful systems.
Both qualitative and quantitative methods were used to query a sample of 645 students from the five PAAET colleges. For the former, focus group discussions were used to explore students' experiences and perceptions of the SIS and improve the questionnaire used to collect data on six crucial aspects of UX for the latter. Taken together, the results from the focus group and statistical analysis of the survey data indicate that the participants had a positive impression of the SIS. As for UX dimensions, the findings revealed that this SIS's perspicuity, stimulation, and dependability were rated slightly higher than its novelty, attractiveness, and efficiency. This suggests that the SIS used at PAAET since 2010 can no longer support the new learning models and delivery modes that students require. Students need efficient systems to collect their academic data in easy-to-use visual dashboards with attractive features such as graphical data representation. These results shed light on the need for continuous SIS evaluation and a broad research scope to develop innovative SIS with intelligent functions for novel activities. Such features enhance students' interactivity and productivity, which encourage their academic success.
This study's results are interest to usability experts and those studying user behavior and practical uses of interactive systems. A poorly designed user interface, ineffective mobile experience, and lack of service availability can turn the SIS into a source of frustration. However, an effective UX earns users' interest and, most importantly, enhances their productivity. The present study provides insight into PAAET's SIS specifically and, in general, highlights the need for educational institutions to perform regular SIS UX evaluations, such as the one illustrated here, which proved to be a valid and reliable way of generating context-specific recommendations.
Future work should focus on designing and implementing intelligent SIS. Intelligent services using adaptive, knowledgebased feedback creates a personalized experience that accommodates individual needs. Innovative SIS also provides what-if scenario analysis, tracks student data trends, provides students with insight, and demonstrates academic progress. Also, the mobile experience with SIS must be reliable, as this interface is only gaining relevance, and some students had negative experiences when using PAAET's SIS on their mobile device. Moreover, redesigning the system according to new creative and innovative approaches and using a more attractive layout injected with stimulating elements that render the interface more user-friendly would enhance the SIS's look and feel while also improving its efficiency and efficacy.