A Framework of Outcome-based Assessment and Evaluation for Computing Programs

—This paper is to present a framework for student outcome-based assessment and evaluation, including the process and detailed activities leading to continue assessment of the successes of an academic program which is essential to its sustainability. Moreover, this paper provides a survey of the literature that reviews the different means of assessing and evaluating an academic program together with the critical performance metrics which aid in quantifying such evaluation. The presented framework is implemented on the Information Technology program over a course of five years. The paper provides empirical insights about how careful implementation of the presented framework enabled the College of Information Technology in Ahlia University to achieve outstanding results in quality assurance and to be ABET accredited. The results of the implementation prove the effectiveness of the framework in improving the student performance and the program. This paper fulfils an identified need to study how student outcome-based assessment and evaluation model enables an academic institute to foster quality assurance instead of relying on ad hoc practices which might lead them to trial-and-error approach. The presented framework could be followed by other institution aiming for international accreditations.


INTRODUCTION I.
The College of Information Technology (IT) of Ahlia University has a clear academic planning framework to fulfill the aims and objectives of its academic programs. The academic planning framework is transformed into an annual college operational plan that is derived from the strategic plan of the University. The academic planning framework for Information Technology Program comprises three perspectives. One is the study plan that has to be carried out in order to execute the program successfully to the students enrolled in the program. The second perspective is the Program review. The purpose of the quality program review plan is to apply measures that ensure the quality of delivery, assessments, and contents of the program. Third is the college operational plan that is aligned with university strategic plan and is meant to conduct activities/tasks that provide continuous support to the successful and smooth execution of the program.
The Information Technology (BSIT) program, as any other academic program, leading to an academic degree, is based on a curriculum constituting of a combination of courses and relevant activities that are organized for the achievement of some learning outcomes defined by the offering institute/ university. In this regard, the program shall have well defined Program Educational Objectives (PEOs) that leads to fulfilling the mission and goals of both college and university. These objectives should:  be broad statements which express what graduates are expected to attain after few years of graduation [1], [2];  serve the missions of both college and university;  satisfy program constituencies needs.
Furthermore, the set of Student Outcomes (SOs) of the program should be defined. These SOs describe what students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge and behaviors that students acquire as they progress through the program. There are six SOs defined by Accreditation Board for Engineering and Technology (ABET) [1].
At the level of courses of BSIT, students' performance is measured by the achievement of relevant course level outcomes. These course level outcomes are called Course Intended Learning Outcomes (CILOs). The CILOs are measured against the performance indicators set at IT program level. These performance indicators are generated from the student outcomes defined for the program. CILOs are satisfying the standards set by Bahrain National Qualification Framework (NQF) of the level 8. The courses in BSIT program have up to thirteen (13) CILOs that are required to be achieved by the students in each individual course. The achievements of the CILOs in all the courses translate the achievement of student outcomes for the program. These CILOs are related to the Knowledge and Understanding (A1, A2, A3), Subject-Specific Skills (B1, B2, B3), Critical-Thinking Skills (C1, C2, C3), and General and Transferable Skills (D1, D2, D3, D4) that students acquire as they progress through the program.
The program assessment and evaluation process should take place at different levels, including PEOs, SOs and CILOs. This ensures having a comprehensive and effective process in place. Regarding program assessment, it is the practice of identifying, collecting, and preparing the data needed for the evaluation. Regarding evaluation, it could be viewed as the practice of interpreting the data collected through the different assessment means to find the attainment of the SOs [3].
The objective of this paper is to thoroughly present and document a framework based on SOs assessment and evaluation that could be adopted by any academic program for the purpose of continuously assessing its success level. Thus, this paper presents the well-defined procedure and detailed activities comprising the framework developed by the College www.ijacsa.thesai.org of IT of Ahlia University for assessing and evaluating student outcomes of BSIT program-that are derived from ABET SOsto fulfil the college mission and goals, achieve program objectives as well as achieve the prescribed graduate attributes by attaining the student outcomes. This framework is based on the longitudinal methodology followed while applying SO based assessment and evaluation model on BSIT program over a course of five years. It is intended to remain valid even if ABET redefined it's SOs. The findings will guide other higher education institutions to continuously evaluate and improve their academic programs, achieve outstanding results in quality assurance, and to be ABET accredited.
The remainder of the paper is organized as follows. Section II presents a detailed survey of the literature highlighting different means of assessing and evaluating academic programs together with the performance metrics in use. Section III provides a comprehensive overview of the proposed framework followed by the College of IT in Ahlia University. Section IV discusses the results of implementing the framework over five academic semesters. Section V lists the challenges faced by the college while implementing the framework and the college actions in overcoming these challenges. Last section presents the conclusion.

Survey of the Literature A.
Literature is rich with studies on the different approaches of program assessment and evaluation. One of the common approaches is curriculum mapping. It explores the links between curriculum content to be delivered and the learning outcomes. This is done by inspecting the relationship between the intended, delivered and received curriculum. The intended curriculum could be examined through the courses' syllabi and materials prepared by faculty members, which contain the learning objectives for each course. The taught curriculum could be examined through interviews with faculty members teaching/ coordinating the courses. The received curriculum could be explored through focus group interviews with students. This approach suggests to use student learning outcome as the base of curriculum development and assessment by treating it as the basis for formative and summative assessments. It considers when, how, and what is taught, as well as the assessment measures in use to gauge achievement of expected student learning outcomes. The strength of this approach is that it aids in identifying whether the intended curriculum and courses' materials has been actually taught and has been actually learnt by students. Moreover, it demonstrates the links between the main components of the curriculum, which are: learning outcomes, learning opportunities, content, and assessment. Furthermore, it studies the curriculum from multiple perspective. On the other hand, this approach has some drawbacks. The main one is being subject to recall bias from faculty members and students. Added to that, it is impossible to determine the extent to which topics were reinforced in the curriculum [4].
Another approach [5] is presented for the ABET-accredited program. Their strategy is based on the assessment and evaluation of their program at both course and program levels. Their process started by defining the vision and mission of their program that serves the university vision. Then they established PEOs that leads to both university and college mission. That was followed by setting the students outcome based on an ABET accreditation body and American Society of Civil Engineers. These SOs are expected to be gained by the students from the different courses constituting their curriculum, by attaining the prescribed Course Learning Outcomes (CLOs). The assessments and evaluations of PEOs, CLOs and SOs were essential components in the assessment of the whole program which leads to decisions and actions that improve the program. In this regard, they mapped CLOs of each course to the most relevant SOs-taking into consideration that the least number of mapped SOs with CLOs is the betterand documented that in a mapping table. It worth to mention that weighted numbers from 1 to 5 were used in the mapping, where the weights represent the strength of each CLO in fulfilling the SO. Moreover, the course credit hours are also taken into consideration to determine how strong the course contributes in attaining the relevant SOs. That was computed using the following formula: Credit hour assigned for each SO= [(total weights of each SO/ Sum of weights) x course credit hour] (1) Due to the ever-increasing importance of outcome-based models, proofed by demanding such model by many accreditation bodies such as: ABET, International Engineering Alliance (IEA) and National Commission of Academic Accreditation and Assessment (NCAAA) [6]-many research works were devoted to shed the light on success stories of adapting this model in assessing different academic programs and getting international accreditations. Some other works were focused on the implementation of a particular stage in the model. Following are some examples.
A complete framework has been proposed in 2019 [7], [8] which is built on set of sustainable practices to advance quality assurance on outcome-based education systems and ensure delivering effective academic program by higher education institutions. It is worth to mention that this framework guided their four programs to successful ABET accreditation. The framework consists of four main types of activities which are: strategic planning, educational practices and strategies, assessment and evaluation and continuous improvement. These activities lead to strategically aligning the program outcomes with the market needs, delivering relevant skills through educational practices, evaluating the attainment level of graduating students and defining remedial actions to improve attainment, in case any deviations in performance were observed in the assessment results.
This framework, as recommended by many international bodies such as ABET with respect to student outcome centered approaches, requires setting measurable learning objectives of the academic programs, measuring those objectives with appropriate assessments to assess and evaluate the effectiveness of the academic programs, as well as having a continuous improvement process to identify the weaknesses in the programs and rectify it quickly.
With respect to the assessment and evaluation phase, they try to make the process comprehensive and at the same time lightweight to avoid extra load on faculty. They used a www.ijacsa.thesai.org combination of direct and indirect assessment methods. The direct assessment was through faculty members and the taught courses as well as an exit exam. With respect to courses assessments, SOs were assessed through formative assessments in the entry level courses and summative assessments in the exit level courses. The formative assessments gave warning signs indicating that a problem is there which needs to be resolved to improve the attainment level. Furthermore, faculty are informed at the beginning of the semester whether their course assessments will be used for program assessments to aid in planning their courses assessments. Courses to be covered in program assessments should have standardize rubrics to assess the attainment of the performance indicators. Such rubrics helped faculty in reducing the variations in assessment process. A program quality assurance team collaborates with faculty to assure that assessment questions intended to assess a particular SO are in line with the rubric. With respect to the exit exam, students were required to do an exam at the time of their graduation, where each question was designed for assessing a specific performance indicator. However, the students did not solve the exam seriously as it did not have any academic weightage. Thus, it was decided not to use the exit exam as an assessment tool again. Indirect assessments allowed them to hear the voice of other stakeholders through surveys and meetings with external advisory board. Jalil in 2019 [9] shared the experience of Energy and Renewable Energy Program in Electromechanical Engineering Department in University of Technology-Iraq in measuring SO attainment while preparing for ABET accreditation. He detailed the process of measuring SO attainment for the four years' work in the program. Although, author frankly stated that attainment of student outcomes is a challenging task for all Iraqi Universities, ABET approved the adopted process as an appropriate way of measuring SO attainment. The Electromechanical Engineering Department started adopting the process of SO assessment in 2014, in which it focused on final exam grades in the assessment and evaluation. After two years, it changed the policy, after consulting ABET experts, and started considering the grades of different course assessments. To achieve that the department put the assessment plans; covering assessments methods, their mapping to CLOs, and the mapping between CLOs and SOs; in the course syllabus at the beginning of the year. Here it is worth mentioning that the department adopted many to many mapping between CLOs and SOs. Faculty members shall prepare the prescribed course assessments with detailed rubrics and performance indicators. Moreover, faculty members shall decide the target to be achieved by the student in order to consider the SO attained for each course, taking into consideration that target shall be above 60% since 50% is the pass degree set by the ministry. After grading course assessments, the average grade of the student is compared with the target and only if it is above the target, the corresponding SO is considered met. In case of an SO not attained, an action plan should be developed to improve the attainment. At the end of the year, the department prepares assessment reportsdetailing assessment results, visual graphs, and the summaryand uses these reports in the annual review. The analysis over the year depicted that SO attainment was improved when they relied on different assessments with detailed rubrics rather than relying on final exam grades only. Moreover, adopting problem-based learning by many faculty members in different courses helped in improving the attainment results. However, it was mentioned that attainment level still needs to be improved in order to qualify the program for ABET accreditation.
In [6], one of the stages of an outcome-based model was spotlighted, which is data preparation. The author presented a coherent vision of a systematic approach of data preparation that aides in collecting the required data (i.e. assessments' data that are mainly used for SOs evaluation as well as program's data that talk about the state of affairs of multiple stakeholders such as students, faculty members, facilities and institute) needed for the accreditation visit by different accreditation agencies. His approach started by identifying the different tasks; which were 23 tasks, to be accomplished in order to satisfy ABET requirements. That was followed by mapping each task to the required resources/ committees, from 7 well defined resources, and defining an interface between the different resources, which define explicit communication between their inputs and outputs. The effectiveness of this approach was illustrated by preparing data-to be used for accreditation-that revealed the real image of the program. Furthermore, it increases awareness, among faculty members, of the importance of such stage in conducting a successful assessment of a program as well as their roles and responsibilities. This led to reducing the load on the program coordinator and smoothed the process of getting the required data from all the faculty members.
Another effort was made in [3] that focused on the design and implementation of performance metrics for successful evaluation of assessments data. This work was made due to the importance of such performance metrics in quantifying the achievement of both program objectives (at program level) and student outcomes (at both program and course levels). The performance metrics were attainment, student achievement and x-th percentile. These metrics with the formulas behind them were tested and their results were analyzed on a sample course. The analysis revealed that getting high values of one metric did not lead to high values of the other metrics and their values depended on the scored marks distribution. They generalize their findings by stating that students' achievement is often less than SOs attainment for mean value which is less than the average marks that is in turn less than the passing threshold for uniformly distributed marks.

Summary and Research GAP B.
The following Table I summarizes the literature review and provides a critique to each of the research works mentioned above: The above summary indicates that there is a need for a framework that is not subject to recall bias from its main constituencies. Moreover, it is desired to be based on SOs assessments and evaluation to facilitate the accreditation by many international accreditation bodies. Furthermore, it shall have a systematic approach of aggregating data from different sources. In addition, should provide a comprehensive assessment to the entire program's SOs using a wide range of assessment tools, such as: courses' assessments, surveys, major projects, just to mention a few. Added to that, the framework www.ijacsa.thesai.org shall have a precise mean of computing the attainment level of each SO, to be compared with an absolute metric goal that is decided by the offering college / department based on the collective experience of multiple faculty members and the market need. -Lightweight (to avoid extra load on faculty) and comprehensive process (Applied on all courses as well as exit exam). This was achieved by treating assessments in entry level courses as formative and assessments in the exit level courses as summative one.
The lightweight assessment process was effective in assessing all the program SOs. However, it was notices that Exit exam was not a reliable assessment tool since there are no academic weightage on it and students do not solve it seriously. Thus, future evaluation process will be limited to courses' assessments. -It adopted many to many mapping between CLOs and SOs.
-The metric goal is different from one course to another determined by the instructor (but should be above 60%).
SO attainment was improved when they relied on different assessments with detailed rubrics rather than relying on final exam grades only. Moreover, adopting problem-based learning by many faculty members in different courses helped in improving the attainment results. However, it was mentioned that attainment level still needs to be improved in order to qualify the program for ABET accreditation -Metric goal should not be determined by an individual (i.e. faculty member) but would be better to be determined by the offering department or college taking into consideration different faculty opinions and experience to reduce the possibilities of human errors.

Rashid, n.d., 2021
Student outcome based model Focus on one stage of student outcome based model which is data preparation by identifying required tasks and mapping responsible committees or needed resources to them.
Ease in collecting data that revealed the real image of the program from faculty members and reduce the load on the program coordinator A great effort in coming up with a systematic approach of aggregating the data from different sources smoothly. This general approach could be used by any institute while collecting and preparing the data to be used for accreditation purpose getting high values of one metric did not lead to high values of the other metrics and their values depended on the scored marks distribution. They generalize their findings by stating that students' achievement is often less than SOs attainment for mean value which is less than the average marks that is in turn less than the passing threshold for uniformly distributed marks Passing threshold should not be relative to the students score. It should be an absolute value determined to ensure that attaining the student outcome will lead to satisfying the market need. www.ijacsa.thesai.org PROPOSED FRAMEWORK III.

Development of PEOs and their Mapping to SOs and A. CILOs
The outcome-based assessment framework proposed started by defining the Program Educational Objectives (PEOs) for the BSIT program, taking into consideration that educational program must be regularly reviewed, assessed, and evaluated [7], [10] for many reasons, including: 1) Meeting educational standards and measures of quality: This award the program a recognition of its quality, integrity and performance which lead to receiving confidence from educational community and the public [5].
2) Ensuring that PEOs are based on the program stakeholders needs as well as assessing the degree of attaining these PEOs which is an important sign of the level of success achieved [5].

4)
Ensuring the sustainability of the program [11].
The final list of PEOs was discussed and approved by the main program constituencies (i.e. faculty, alumni, external advisory board, and employers); which is: 1) Exhibit the relevant skills and knowledge for pursuing the IT career in industry including corporate as well as government sector.
2) Pursue life-long learning leading to entrepreneurship, research and development.
3) Contribute to the society through their ethical and professional norms by demonstrating them in IT professions.
These PEOs are supposed to be fulfilled through the different courses in the program curriculum. Table II presents the mapping of ABET six SOs to BSIT three PEOs.
Each course has a set of CILOs that should be attained by the students upon completion of the course. Each student outcome is translated to the relevant performance indicators to assess the student outcomes within a program. To assess the student outcomes for the program, each performance indicator is mapped to the relevant CILOs of that particular course. Table III shows the complete view of the BSIT program student outcomes in relation to performance indicators as well as their one-to-many mapping to CILOs, assuming equal weights across all CILOs mapped to a particular SO.  CILOs A number of data collection methods as well as assessment tools have been defined to facilitate measuring the achievement of both CILOs and SOs and consequently PEOs. After that comes the stage of utilizing the results and feedback in continuously improving the program with its learning and assessment processes and achieving high quality standards.
The proposed approach has short term (every academic semester) and long term (every three years) processes. It is carried through three main stages, which are: 1) Data collection and preparation: Data is gathered from various potential-internal and external-sources that cover all the program stakeholders. The primary sources are: a) Faculty developed assessment: A faculty can use variety of summative and formative assessment to evaluate the student during the semester. The grades of each question in the summative assessments need to be recorded for all the enrolled students in a course. This is done by filling the CILOs Evaluation excel sheet by the faculty member. This CILOs Evaluation excel sheet needs to be submitted as part of the course portfolio to the college by the end of the semester. b) Internship feedback: each student is required to go through industrial training program. During internship, the assessment of the students is done by their industrialsite-and academic supervisors. The site supervisor evaluates the student twice; after the completion of the first month of the internship period and after the completion of the internship period. The evaluation is done by filling the mid-evaluation written questionnaire form and final evaluation written questionnaire form. The mid-evaluation form consists of 10 questions; with 5-points scale; and a section to write comments. The final evaluation form has the same set of ten questions plus a space for the evaluator to comment on the strength / weaknesses of the intern as well as provide recommendations on preparing students for the workplace. In similar way, the academic supervisor evaluates the student after the completion of the internship period by filling a written questionnaire form with 5points scale. Furthermore, a summary report of all the student undertaking the industrial training will be prepared by the college internship coordinator and submitted to the college by the end of the semester. c) Major project examination and exhibition: Students in their major project course are examined orally after submitting their written reports. The reports and oral exam aids in performance evaluation by the examination committee, which consists of two faculty members and an external examiner from the industry. Ready assessment forms are prepared to help in assessing and evaluating the students. Moreover, an exhibition is arranged to give the students an opportunity to present their developed projects to various businesses. Many people, including experts from IT field, attend the exhibition and spend ample amount of time in examining and evaluating the projects as well as the students' skills. Their evaluations and comments will be collected through a survey prepared by the college. d) Written Surveys: The College collects feedback from variety of stakeholders and makes use of that in assessing and making informed decision. A third party is responsible for conducting the written surveys, collection and collation of multiple stakeholders' feedback that are passed for the colleges for further action. The main written surveys are: i) Alumni survey: This felicitates graduate's satisfaction with the program and their learning experience.
ii) Senior exit survey: This reveals students' satisfaction level with respect advising, teaching and learning, administrative support and facilities. Moreover, it prompts students to overall rate the program.
iii) Employers' satisfaction survey: This enables the college in receive employers of the graduates' feedback regarding graduates' job performance, thinking skills and character.

2) Data collection and preparation:
The success of the program is assessed by the level to which it satisfies the university and college missions and goals. Since the PEOs were derived from the college mission which is in line with the university mission, assessing the PEOs leads to assessing the whole program. PEOs are assessed through the SOs and CILOs mapped to them.
This assessment process is carried at both course and program levels via various assessment tools. Some of these tools are directly assessing SOs and CILOS while others are indirectly assessing them. In case of indirect assessments, a mapping should be done to facilitate assessing CILOs and SOs. The level of CILOs/SOs attainment is compared with a metric goal set by the college. If the attainment level is below the metric goal, then it will be considered as not attained.
These assessment tools are: a) Faculty developed assessment: It is a direct assessment tool carried by faculty teaching courses comprised by the curriculum. A Faculty member assesses students' attainment of CILOs, and in turn SOs, through different course assessments. The faculty will be recording the students' grades in CILOs/ SOs evaluation excel sheet for each course. This sheet details the summative assessment methods used in assessing the student outcomes. Because this serves as quantitative analysis, questions are used as the basic units of computation of assessment. That's why, CILOs/ SOs evaluation sheet records all the students' grade for each question in each assessment and provides a clear mapping between each question in the course assessments and CILOs/SOs. Here, it is worth to mention that each question could be mapped to maximum two CILOs/ SOs. In addition, CILOs/ SOs evaluation sheet provides a summary of the results, indicates the level to which each of the CILO/ student outcomes is being attained, and determines whether the CILOs/ student outcomes have been met or not by comparing their average percentages with the predefined metric goal. This leads to absolute student achievement evaluation since the metric goals is fixed to a target value. It is noteworthy that in case of having students from other academic programs in the courses, www.ijacsa.thesai.org separation must be done during the assessment phase to ensure that only IT program's students are considered in the assessment and evaluation. b) Internship written questionnaires and forms: This is a direct assessment tool since the CILOs assessed by each evaluation form / submitted report is clearly written in the internship course syllabus. It worth to mention that 50% of the student final grade is taken from the total score given by the site supervisor in the two evaluations. However, only 10% of the score given by the academic supervisor will be counted. The remaining 40% of the student grade is based on the evaluation of the three reports submitted by the student during his internship period and after the completion of the internship. Furthermore, CILOs/SOs evaluation excel sheet will be prepared for this course to be taken into consideration in the program evaluation process. c) Written surveys: These are indirect assessment methods. In alumni survey, "alumni experience with the program" section was mapped to SOs and thus used in the assessment. In senior exit survey, "program overall rating" section was mapped to SOs and used in the assessment. In employer satisfaction survey, SOs were clearly listed to seek employers feedback on them. In addition, "employee learning outcomes and skills" section was mapped to SOs to aid in getting comprehensive assessment.
The aforementioned assessment tools are carried out with different frequencies. Every academic semester, all the direct assessments will be carried out in addition to the senior exit survey. Other surveys are conducted every year.
At the end of each academic year, the result of all the assessment methods will be compiled together, as shown in Fig. 1. That is done by giving weight to each of the assessment tool. In this regard, each of the survey was given 10% and by that we assure balance consideration of the stakeholders (students, alumni and employers) and 70% was given to faculty developed assessments which represent the main direct assessment method. This will give us the percentage of attaining SOs in the year. 3) Evaluation: At the end of each academic semester, IT College holds meetings to discuss thoroughly the assessment results, including: SOs achievements and surveys summary reports. With regards to SOs achievements, it gets evaluated on course level as well as on program level. In case of course level evaluation, the attainment of each SO is based on the attainment of CILOs that represent PIs mapped to it. An SO/ CILO is considered attained, if the average of the students' grades of questions mapped to SO/CILO is not less than the predefined, absolute, metric goal which is 70%. If any of the SOs is not attained, the course coordinator/ instructor will be requested to provide a reason behind this pitfall and formulate an action plan to be carried on the next semester to improve the SO attainment level and overcome this downfall. On the next semester in which the course is offered, the IT College quality coordinator will follow-up with the faculty member to check the attainment level of those SOs that were not attained earlier.
If any SO is still not attained, the reason should be provided as well as another action plan to be placed to be implemented on the following semester. Table IV presents some examples of SOs not attained on the course level and the process of monitoring them by the college. This practice had high impact on improving the teaching methods since faculty members were encouraged to evaluate the effectiveness of their teaching and adopt new techniques that were not used previously in an aim to improve SOs attainment. Moreover, some faculty tried to boost the attainment by revising and adjusting the lesson plan and devoting more time for topics/ concepts that were not comprehended by students. In like manner, assessment methods had been improved because faculty members were motivated to use verity of assessments in assessing each of the SOs. This ensures getting reliable assessment results, because, even if one of the course assessments was not effective in assessing a particular SO, that will be overcome by other course assessments.
The same process is applicable for evaluating SOs on program level; by considering all the courses with CILOs corresponding to its PIs.
Achieving good results demonstrates that the PEOs are based on the needs of the program's various constituencies. In addition, it demonstrates the high competence of the program graduates in their professional career as well as the high level of their employer satisfaction. All the assessments and evaluations results and findings are well documented in the college meeting minutes as well as the annual report produced and maintained by IT College. The report clearly states the pitfalls/ deficiencies, justification of each, areas to be improved, action to be taken in order to improve the program outcomes, meet high quality international standards and ensure its sustainability. This aids in planning how to overcome the existing deficiencies in the near future. Such documentation and reporting reflect the ongoing evaluation in place leading to continuously improve the program effectiveness.
RESULTS AND DISCUSSION IV.
The longitudinal methodology opted in this research enabled in assessing SOs achievement and evaluating BSIT program over the last five years. Data was first collected at the outset of the study and then gathered, repeatedly, during the course of the study (i.e. five years). That facilitates observing changes in SOs achievements over time. Fig. 2 provides empirical insights of the performance of the College of IT after adopting the proposed framework. It summarizes the results of the assessments approach over 5 academic semesters. It illustrates the extent to which each of the SOs is being attained using the weighted combination of direct and indirect assessment methods.
The student outcomes achievements were maintained high over the last five semesters. This comes as a result of the careful implementation of the proposed framework that enabled the College of IT in having a systematic assessment and evaluation process, leading to achieving outstanding results in quality assurance and being ABET accredited. Moreover, there are number of notable routines and practices that did support the presented framework in achieving these satisfying results. Among these practices are advising and counseling that provide customized and personalized advice to individual students guiding them in defining and achieving their academic, career and life goals. That enabled the college in maintaining good student ratio. Furthermore, different workshops are regularly conducted by the college to qualify faculty members with new teaching and assessment methods. Added to that, reviewing the courses' syllabi, routinely, at the beginning of every academic semester and ensuring having upto-date textbooks brings new advancements in IT into the IT curriculum and confirms that the program is up to date.
The results of the implementation proved the effectiveness of the proposed framework together with other educational practices in improving the student performance and the program.   Getting such outstanding results does not mean that the IT program does not need continuous improvements. In this regard, the College of IT believes that curriculum development is a dynamic process that ensures having an up-to-date program in line with the rapid advancements in Information Technologies and in compliance with ABET requirements in case ABET changes any of their criteria or requirements.

CHALLENGES V.
The College of IT faced number of challenges during the course of implementing the proposed framework. Following is the list of challenges together with how did the college overcome each:

Balancing between NQF Requirements and ABET A.
Requirements Bahrain National Qualification Framework has a set of standards and requirements to be satisfied by an academic program in order to endorse its associate qualification and be placed on a specific NQF level (from level 1 for Access1 to level 10 for Doctoral Degree).
BSIT satisfies the standards set by NQF of level 8 [12]. However, in order for BSIT to be accredited by ABET, the program should also satisfy the requirements of ABET. To achieve that, the college came up with a mapping of PEOs, SOs and CILOs, as shown in Table I and Table II.

Having Students from Different Majors in BSIT Program B.
Courses Some of BSIT program courses are offered for students from other colleges (such as Engineering College and Business College). However, calculating the attainment of SOs for BSIT program requires considering IT students only. In this regards, the college decided to separate the result of IT students from other students and simply consider IT students' grades in the assessment and evaluation. In case of having few IT students (i.e. less than four) registered in the course, the results will not be considered in the evaluation of the program.

Online Course Delivery during COVID-19 Period C.
Although classes were delivered online during the pandemic, but all the courses content and CILOs were covered and assessed. Even the practical components of the courses were delivered through virtual labs. Thus, the process of assessment and evaluation of SOs were not affected.

Market Needs and ABET Requirements D.
Market needs were analyzed and considered while designing BSIT program as well as formulating its goals and objectives. BSIT graduates should be qualified enough to fullfil market needs by having distinguished and desirable skills and knowledge. Fortunately, the proposed framework secured the alignment of program outcomes with market needs by ensuring delivering the desired skills, evaluating the attainment of graduates and taking a remedial action in case any of the SOs was not attained.

Effect of the Correctness of Mapping of SOs to CILOs on E. the Reliability of the Evaluation Results
It is well known about SO based model that the quality of its evaluation result depends on the quality of the mapping between CILOs and SOs. Thus, program coordinator (who creates the mapping between CLOs and SO) and instructors (who map assessments' questions to CILOs and SOs) shall be qualified enough to do correct mapping to ensure having reliable results at the end. Due to the highly qualified faculty members in the College of IT in Ahlia University, the obtained results were reliable and trustworthy.

Overloading Faculty Members during the Data Collection F. and Assessment Phases
Faculty were overwhelmed with their responsibilities in the data collection and assessment phases. However, it has been www.ijacsa.thesai.org planned to reduce that in the near future and have a lightweight process by selecting some of the courses and their assessments to be considered in the program assessment and evaluation, while other course assessments will be considered as formative one. This should reduce the load on faculty while maintaining good evaluation result that aids in informed decision making and continuous improvement of the program. CONCLUSION VI.
The student outcome-based framework presented in this paper has been adopted by College of IT in Ahlia University for the last five years. The vast amount of data collected by the college during this period, boosts the robustness of this framework in extracting continuous improvement requirements. Moreover, number of assessment tools were in use to measure the achievement of CILOs, SOs and in turn PEOs and their results were utilized in making informed decisions regarding continuously improving the program with its learning and assessment processes, achieving high quality standards as well as assuring the sustainability of high quality education.
Although some challenges were faced during the implementation of the proposed framework, this framework helped the college in improving the academic program. Other educational practices supported this framework such as: academic advising and career counselling. The success is reflected by improving the curriculum through adding new courses, removing courses, and modifying the content of some courses based on the market needs, international standards, constituencies feedback and the new trends in IT. Moreover, enhancing current teaching and assessment methods as well as adopting new one is another sign of improving the program that promotes positive assessment results. Furthermore, achieving distinguished and honorable results-such as improving students' performance, attaining high level of graduate attributes, achieving high quality standards, being accredited by ABET and satisfying various stakeholders-are clear indications of the success and improvements in the program.
Despite the fact that the College of IT adopted SOs defined by ABET as SOs for BSIT program, the proposed framework will still be applicable even if ABET re-define their set of SOs. Moreover, it is also applicable to other academic programs.