An Effect Assessment System for Curriculum Ideology and Politics based on Students’ Achievements in Chinese Engineering Education

—The curriculum ideological and political education (CIPE) has caused the attention of China’s leaders and state departments in Chinese education, but its effect assessment is still an open issue should be address for the efficient and effective implementation of CIPE. The engineering education conception has been widely adopted in Chinese higher education, in recent years, due to its effectiveness. Therefore, in this paper, under the background of Chinese engineering education, we study the CIPE effect quantification. We propose a CIPE effect assessment system for higher education, and a CIPE effect quantitative method based on the achievements of graduation requirements for each student. The proposed system provides visualization information of achievements and CIPE effect for students and teachers. This helps students to locate themselves in their major learns, and teachers to continuously improve their teaching methods.


I. INTRODUCTION
Recent years, Chinese national leader and institutions, e.g., State Department and Education Ministry, put a high value on the curriculum ideological and political education (CIPE), to raise the ideological and ethical standards of Chinese students. They require all educators must pay equal attention to the professional ability training and the ideological and political education (IPE). In addition, IPE must be integrated into all aspects of education, instead of only relying on only a few IPE curricula. The Chinese Ministry of Education and other nine national-level government bodies jointly issued the official document of Work Programme for Fully Pushing Forward the Construction of "Great Ideological and Political Course" in July 2022 [1], which plans to fully mobilize social forces and resources for CIPE in all universities, middle and primary schools in China.
Therefore, many Chinese researcher study on how to organically integrating IPE into the professional education in various subjects, e.g., computer science [2], [3], [4], mathematics [5], [6], agronomy [7], and so on, to the isolated island phenomenon in IPE [8], and develop the world-class modern education with Chinese characteristics [9]. For example, Lu [10] applied new media network platform and block chain technology to design a CIPE information exchange approach, to improve the sharing efficiency of IPE resources. This work also used the bad information screening technology to filter out negative information. Liu and Ni [11] exploited SWOT analysis method qualitatively analysing the inherent strengths and weaknesses as well as the external opportunities and challenges of the IPE politics employed by universities and colleges, currently, in HeiLongJiang province of China. Lin et al. [3] introduced a strategy for integrating IPE into the artificial intelligence course, and designed the IPE achievement gaols of the course, based on the idea of outcome-based education (OBE). Zhang et al. [12] studied the combination of CIPE and Chinese engineering education accreditation, and integrated IPE into various educational phases including the educational goals, graduation requirements' indicators, the course system, and teaching designs. These works studied the integration strategy of CIPE in some aspects, but didn't consider to evaluate the effectiveness of their strategy in improving ideological and political achievements for students.
There are several works on the effect assessment of IPE education. Zhang [13] proposed to exploit industry education integration for CIPE education, which combined field practice and classroom teaching, to improve the mental health of students, by increasing the attraction of IPE education. Tang [14] studied the CIPE education reform for innovation and entrepreneurship education, to improve the self-efficacy of fresh graduates in their job applications. These works used grouping comparison to evaluate the effectiveness of CIPE strategies. Chen and Yu [15] used the principal component analysis technology to reduce the information redundancy of CIPE assessment criteria, and performed the weighted sum on the principal components to establish the evaluation model for CIPE education effect. Similarly, Zhang [16] employed the clustering algorithm to find the similarities of data indices, and built a CIPE evaluation data model by assigning the weight coefficient of each index class produced by factor analysis. Mo et al. [17] utilized triple helix model to combine the assessment data from three subjects, school, enterprise, and government, and established a fuzzy synthetic evaluating model for evaluating the CIPE effect. Ding et al. [18] used the back propagation neural network to evaluate the the CIPE teaching quality of teachers, with the data scored by students.
This work employed the traditional idea of teacher-centered instead of student-or learner-centered education. Zhang et al. [19] applied deep learn model for analysing the CIPE teaching quality, with multi-source data from on-line and classroom teaching, and produced the deficiency of teaching. Guo [20] designed an evaluation system of CIPE effect based on the authenticity evaluation theory for vocational education. All of these above works achieved the data through scoring subjective questions, leading to objectivity of CIPE effect assessment. Lv et al. [21] developed a data management system for IPE, where the evaluation of IPE effect is same to that of usual curricula, where teachers scoring.
To our best knowledge, there is lack of objective and quantitative evaluation method for CIPE effect. Existing works evaluated results of CIPE, ignoring the dynamic of the political and ideological status for each student. Therefore, in this paper, we study on the quantitative evaluation strategy for CIPE effect, based on the dynamic of achievements of students during their school years. The achievements of every student is the progress that the student reaches the graduation requirements specified in its specialty personnel training program (PTP). The graduation requirements are the main goal that undergraduates pursue in their four-year university or college life, in the context of engineering education with the idea of outcomebased education (OBE), in China. The graduation requirements are stable during students' school years. Thus, the fluctuates of students' achievements can reflect their IPE states. Such as, an improvement of achievements imply a better IPE effect (selfdiscipline) for a student. Our work complements to existing works, and helps to provide a more comprehensive assessment information of CIPE effect.
The rest of this paper is organized as follows. In Section II, we present preliminary knowledge on the Engineering Education. Section III illustrates our CIPE effect assessment system, and Section IV details the assessment method for quantifying the CIPE effect based on achievement data. In Section V, we illustrates some application cases for students and teachers, on the CIPE effect. In Section VI, we conclude this work and present our future work.

II. PREPARATORY KNOWLEDGE ON ENGINEERING EDUCATION
In 2016, China became the 18th official member of Washington Accord, and the undergraduate engineering degree is mutual recognition among the countries joining the Washington Accord, for certified engineering specialties. This promotes the development of internationalization and modernization of Chinese higher education. Therefore, numerous Chinese universities actively apply for engineering education certification. In 2022, almost 2, 000 university specialties have gained the certification from China Engineering Education Certification Association (CEEAA) [22].

A. Engineering Education Conception
The concepts of engineering education are student-center, outcome orientation, and continuous improvement. There are one kind of eventual outcomes (cultivation goals) and two kinds of intermediate outcomes (graduation requirements and course objectives) in engineering education, As shown in Fig. 1. The cultivation goals are the capacities that students will have after graduating about five years, which are designed and continuously improved based on the developments of social demands and expectations, the school and specialty orientations, etc. Graduation requirements are the capacities that students graduated with, by the college or university education. Graduation requirements are designed on the principle that they can support the reaching of cultivation goals. Course objectives are the capacities obtained by students through various courses directly. The accumulation of course objectives must support the achievements of all graduation requirements. And the course objectives are accomplished by corresponding teaching contents and methods.
As shown in Fig. 1, there are four continuous improvement circles or closed loops in engineering education, as illustrated in the followings.
1) The ultimate goal of engineering education is enabling students to achieve cultivation goals. Thus, the cultivation goals must be designed legitimately, as they decides the rationality of the whole PTP. Thus, when the cultivation goals are designed, their rationality must be assessed by related enterprise, industry, and education experts. When the assessed rationality is low, the cultivation goals are improved based on expert comments. These above steps are repeated for improving the cultivation goals until they are assessed to be rationality. The improving process is conducted periodically, as the social needs are changed with the world development. Usually, a major improvement is made in the cultivation goals every four years, and a minor one is made every two years, in Chinese higher schools. 2) Given the reasonable cultivation goals, the graduation requirements are designed to enabling students to have the ability to achieve these goals after graduation. Thus, the rationality of designed graduation requirements can be evaluated by the reaching degree of cultivation goals. If the reaching degree is low, the design of graduation requirements cannot support the achievements of cultivation goals, and thus need to be improved. These process is conducted when the cultivation goals are changed in the first continuous improvement circle or their evaluation results do not meet expectations. 3) Cultivation goals and graduation requirements are the high-level designs of PTP. Graduation requirements are the achievements that a student get in its about four years of engineering education, by various courses. Therefore, the course system must be built carefully, and designed course objectives need be equivalent to graduation requirements. After completing a course, a student accomplishes corresponding course objectives, and gets closer to corresponding graduation requirements. Thus, if the achievements of graduation requirements are evaluated to be poor for fresh graduates, there is a strong likelihood that course objectives are improper and they need to be improved to guarantee that students meet graduation requirements if they achieve all course objectives. 4) For each course objective, its accomplishment for students is implemented by reasonable designs and  Fig. 1. The core concepts of engineering education arrangements of course teaching contents and methods. When the accomplishment degree is low for a course objective, the corresponding teaching contents and methods need to be improved.

B. Graduation Achievement Quantification
Given from CEEAA, certification standards show that there are 12 graduation requirements including engineering knowledge, problem analysis, solution design/develop, research, etc. To easily evaluating the achievement, each graduation requirement is decomposed into 2-5 smaller requirements, which can help students achieve the graduation requirement progressively and accumulatively.
Assuming that there are total N graduation requirements (g i ) and M course objectives (c j ). The relationship between graduation requirements and course objectives can be represented as a support matrix (W), where item w i,j on line i column j is the support weight/degree of c j to g i . If c j don't support g i , w i,j = 0. For each graduation requirement, the achievement is quantified by the weighted sum of course objectives' accomplishment degrees, as shown in Eq. (1). Where r i and o j are respectively the achievement and accomplishment degrees of g i and c j , and they are both in the range from 0 to 1. And thus M j=1 w i,j = 1, representing course objectives with non-zero weight jointly support the achieving for every graduation requirement. The accomplishment degree of each course objective is obtained by testing in the corresponding course.
All course objectives are tested only when near the graduation, as courses are run on all time periods (e.g. terms) of every student's school life. Thus, the graduation requirements are achieved incrementally for each student in its school life. In this paper, we focus on the fluctuation of graduation requirement achievements to quantify the CIPE effect for each student. Therefore, we normalizes graduation requirement  Fig. 2. The framework of CIPE effect assessment system achievement degrees as achieving ratios in a same value range ([0, 1]) at any time, using following processes.
As course objectives are tested at different times, a student's school life can be divided into M time periods, by these test times. Without loss of generality, the test order is c 1 , c 2 , ..., c M . The jth time period, t j , is the period between the times of testing c j−1 and c j , and we can obtain the accomplishment degree of c j at the end of t j . Then, we define the achieving ratio (AR i,j ) of g i at t j as the ratio of the current achievement degree of g i to the full degree when o j ′ = 1, ∀j ′ ≤ j, as in Eq. (2).

III. CIPE EFFECT ASSESSMENT SYSTEM
The framework of CIPE effect assessment system is shown in Fig. 2. There are mainly six modules in the system, as detailed in the followings, respectively.

M1. Course objective accomplishment entry module
provides entry interfaces (keyboarding, file import, etc.) for various courses' test data to teachers, and formats entered data into course objective accomplishment degree. For example, a course has two course objectives (c 1 and c 2 ), and its test data include scores of two assignments (80% and 75% for a student case), one middle exam (65%), and the final exam (77%). When the accomplishment of c 1 is supported by these two assignments equally, and c 2 by these two exams equally, the accomplishment degrees of c 1 and c 2 are respectively 0.775 ( 80%+75% 2 ) and 0.71 ( 65%+77% 2 ), for the student case.
M2. Achieving ratio calculation module updates corresponding achieving ratios when the accomplishment degrees of some new courses objectives are calculated by M1 for some students, using Eq. (2).
M3. CIPE effect quantification module calculate the quantified data for the CIPE effect for each student, based on the fluctuation and the statistical information of achieving ratios. The quantification method is illustrated in Section IV.
M4. Achievement data visualization module uses some visualization tools to show the achievement data of graduation requirements for every student, including the current achievement degrees and the fluctuations of achieving ratios. This can help the student to know its position in the specialty. For teachers, the module shows the statistical information of achievement data.
In Section V, we will present some cases to illustrate the visualization of achievement data.
M5. CIPE effect data visualization module is similar to M4, which visualizes the CIPE effect data obtained from M3. Some cases will be illustrated in Section V.
M6. Incentives and feedback module recommends incentive strategies to students and teachers according to the historical achievement and CIPE effect data, and evaluates the effectiveness of an incentive strategy by testing whether there is a different before and after performing the incentive strategy. The recommendation of incentive strategies can be implemented by modern artificial intelligent algorithms, which is one of our future works.

IV. CIPE EFFECT QUANTITATIVE METHOD
In this paper, we design a CIPE effect quantitative metric based on the achieving ratio of each graduation requirement according to following principles.
• CIPE effect will be good if the achieving ratio is increased, and opposite when the achieving ratio is decreased.
• The small fluctuation of the achieving ratio can reflect the discipline of the student, and thus corresponds to a good CIPE effect.
• If the rank of a student is risen, the CIPE effect is likely to be good.
Thus, we construct the CIPE effect quantitative metric using following indicators.
• The increment of the two successive achieving ratios. ∆ i,j = (AR i,j − AR i,j−1 ) is the current increment at t j for g i . ∆ i,j can be negative when the achieving ratio is decreased.
• The number of the achieving ratio increasing.
• The number of the achieving ratio decreasing, D i,j = j j ′ =1 (∆ i,j ′ < 0), similar to the previous indicator.
• The standard deviation of historical achieving ratios, is the average of achieving ratios, which is j j ′ =1 AR i,j ′ /j. This indicator reflects the fluctuation degree of the achieving ratio.
The CIPE effect quantitative metric is defined by Eq. (3) based these above four indicators, for each graduation requirement, and the overall CIPE effect quantitative metric is defined by Eq. (4). The greater values these metrics have, the better CIPE effect is.
V. CASE STUDY In this section, we first present cases for illustrating how students and teachers can be benefited from our CIPE effect assessment system by visualization information.

A. Visualization for a Student
For each student, our system first provides its latest achieving ratios for all graduation requirements by a combination chart, as shown in Fig. 3. As shown in the figure, the system provides not only its achieving ratios, but also the average and the best ones of all students, for the student. In addition, the chart highlights the achieving ratios not passing in red color. By comparing its achieving ratios with the average ones or the passing lines, the student can get a clear sight in its position and reasonably plan next goals.
In addition, the CIPE effect assessment system provides the dynamic changes of each graduation requirement achieving ratio for a student, in two forms, as shown in Fig. 4 and 5, respectively. Fig. 4 gives the achieving ratio for the first graduation requirement in all time periods. The system also provides the achieving ratios of others graduation requirements for the student, as same as Fig. 4. From Fig. 4, the student can see its achieving ratios, as well as the average and the best ones. In addition, Fig. 4 gives the suggest of the next goal for the student, according to its current achieving ratio. The suggested goal is set as the average achieving ratio of students with a higher grade than the current achieving ratio. For example, for a student, the current achieving ratio is 0.73. The average achieving ratio of students with achieving ratios between 0.8 and 0.9 is 0.84. Then the next goal is suggested to be 0.84. To give a more clear sight of the fluctuation of the achieving ratio, the system also provides the variations of the achieving ratio for each student, as shown in Fig. 5.
Based on the achieving ratio, the system calculates the overall CIPE effect by Eq. (4), and shows the result by a bar graph, as Fig. 6. By this graph, the system provides the overall CIPE effect, and the average and the best one for every student, which helps it to locate itself in its major learn and CIPE effect.

B. Visualization for a Teacher
For a teacher, the assessment system provides the statistical information on achieving ratio and CIPE effect by boxplot graphs, as Fig. 7 and 8, respectively. From these graphs, the teacher can see clearly the relative difference among achievements and CIPE effects of all graduation requirements. With the knowledge of which graduation requirement has poor or good achievements and CIPE effects statistically, the teacher can analysis the strengths and weaknesses of its teaching methods, and hasten the strengthening and make up for the weaknesses in the future.  In this paper, we study on the CIPE effect assessment for Chinese higher education. We present a CIPE effect assessment system to manage the data of graduation achievements and the CIPE effect assessment of students, and propose a CIPE effect quantification method for the CIPE effect based on the graduation achievements. Our CIPE effect assessment system and method can be helpful for continuous improvements on achievements of students and teaching methods of teachers. To our best knowledge, this is the first attempt for quantify the CIPE effect based on objective data. In the future, we will study on the integration of our method with the subjective assessment for the CIPE effect, to provide more comprehensive knowledge for students and teachers.