Normalisation of Technology use in a Developing Country Higher Education Institution

The purpose of this study is to understand how the use of an online course and lecturer evaluation becomes a normalised way of evaluating courses and lecturers in a developing country higher education institution. Extant literature on course and lecturer evaluations has concentrated on the approaches to evaluating courses, lecturers, and its effectiveness and benefits. However, less attention has been paid to how online evaluations become the medium for lecturer and course evaluation. To address this gap, this study used an interpretive case study approach to collect data through semi-structured interviews, documents and participant observation. Data analysis was conducted using hermeneutics and using Normalisation Process Theory as the theoretical lens. The results show that the online evaluation of courses and lecturers is now a normal practice because of participant’s investment in the meaning of the online evaluation process, their enrolment in the process and the crucial investment of their actions, feedback during implementation, and use of which ensured the normalization. Keywords—Course and lecturer evaluation; Higher Education Institution (HEI); Normalisation; Normalisation Process Theory


I. INTRODUCTION
The purpose of this paper is to understand how the use of an online course and lecturer evaluation process becomes normalised evaluation process in a higher education institution (HEI).Course and lecturer evaluation is the most commonly used method of assessing a course and lecturer effectiveness because it offers important opportunities for feedback and development [1], and has been routinely used in academic institutions to inform curricular change and assess lecturer's performance [2].Whilst course and lecturer evaluations may be paper-based or virtual, it has largely been conducted in class at the end of the academic semester through the use of paper evaluation forms [3] in many developing countries.Despite the limitations of high financial cost, waste of time and problems with analysis the paper-based evaluations are widespread [4], [5] in developing countries.As a result, many HEIs in developing countries are migrating to online evaluations where students use online forms [1], [4].A lot of studies on the course and lecturer evaluation in HEIs [6]- [8] have largely been quantitative with less qualitative studies.Apart from the lack of qualitative research in the area, there exist some knowledge gaps in understanding how the use of online evaluations becomes a normal practice, especially in HEIs in SSA.
Paper-based evaluations have been cited to have some problems such as the vulnerability of lecturers influencing students on the day of the evaluation by their presence or otherwise [9].This is because the presence of the lecturer when the students are conducting the evaluation may create an intimidating environment which may influence what the students put on the evaluation forms.Also, the security of the evaluation form is a problem.This is because lecturers can pick and choose which forms to take forward as part of the evaluation.Unfortunately, very little attention has been given in the literature to understanding how the new ways of evaluating courses and lecturers become the norm.The research question, therefore, concerns how the implementation and use of such new technology can be normalised in HEIs.
The study is conducted in a University in Ghana (Herein referred to as UNID).Ghana was selected for a number of reasons.First, the researchers are Ghanaians and are faculty staff in Ghanaian Universities and believe that their knowledge about the country and the University set up as well as their social networks there could facilitate gaining research access.
The rest of the paper is aimed at how this question is answered using a coalescing of Normalisation Process Theory (NPT) and empirical evidence derived from an interpretive case study approach.The following sections are organized as follows.Section 2 examines the literature on the course and lecturer evaluations.The theoretical foundation and the methodology are discussed in Sections 3 and 4, respectively.Section 5 presents the case study description and the analysis and discussion of the findings are presented in Section 6.Finally, Section 7 concludes the paper and outlines its contribution, implications and suggestion for further research.

II. BACKGROUND OF COURSE AND LECTURER EVALUATIONS
Student evaluations of courses and lecturers are also one of the most controversial and highly-debated measures [42].Nonetheless, they are still widely used and many have argued that there is no other option that provides the same sort of quantifiable and comparable data [40].
Largely, course and lecturer evaluations are used to make personnel decisions in terms of hiring tenure, promotion, and so on and this is based in part on a student's evaluation of lecturer's teaching effectiveness.The qualitative responses are also used as a feedback for lecturers and other teaching www.ijacsa.thesai.orgsupport offices to ensure improved teaching and course development.In [43], author cautions against the use of instruments not specifically designed to provide feedback for this purpose, and that separate instruments should be designed to provide summative and formative feedback, respectively.
Much has been written about the problems with the course and lecturer evaluations.Educational scholars have examined issues of bias and concerns regarding the statistical reliability of evaluations of lecturers and have questioned their ability to accurately gauge the teaching effectiveness of staff.In addition, some have argued that the feedback provided by course and lecturer evaluations does not effectively promote change in lecturer's behaviour.However, a significant majority of researchers consider student evaluations to be a useful measure of the instructional behaviour that contributes to teaching effectiveness [40], [41].
Whilst student evaluations have largely been conducted physically using paper-based evaluation forms, many educational institutions are migrating to online evaluations [15], [38].HEIs are leveraging on the advantages that an online evaluation could bring.This is because student evaluations are seen as a very important yardstick in the retention, promotion and tenure decisions of lecturers in higher educational institutions [43].With this importance, many academic staff is concerned that a migration to an online evaluation may have effects that can change the whole evaluation process.Lower response rates by students have been cited as one of the effects [6].Though there is less research on online course and lecturer evaluations and its implementation in the developing world, several institutions in the developed world have successfully implemented online student evaluations [9], [10] .
Despite the widespread implementation in the developed world many higher educational institutions and academic staff still question their value [11], [12].Several advantages have been cited in the literature for the migration of physical evaluation of lecturers to online evaluations.The quick turnaround of student evaluations is one of the mainly cited advantage.This provides academics more rapid feedback to refine the curricula or the overall educational design [11]- [13] cites the ease for students to write their reflections of the learning experiences on a keyboard than by hand.
The research on student course and lecturer evaluation is widely dominated by literature on students' experiences [9], [13].However, in a recent study by [12] on the migration from paper to online evaluations, it was found that most lecturers still preferred traditional paper-based evaluations.The lecturer's perception was that the paper-based methods resulted in higher response rates.Others have mentioned lower response rates in online evaluations because it involves out-ofclass time and students can be distracted and not remember to fill the form or they may simply choose not to do it [14].Technical glitches in accessing the online forms are discussed in [15], and the issue of anonymity of online responses are discussed in [9], [16].

III. NORMALISATION PROCESS THEORY
NPT provides a framework for understanding how a new intervention becomes or not becomes part of normal practice [17] by examining how social processes affect the new ways of working [18].NPT seeks to understand the dynamics of embedding a practice in an institution as part of implementing, integrating and using this practice to influence business processes [19].NPT provides a set of tools that explains the processes through which new or modified practices of thinking, enacting, and organizing work is operationalized in institutional settings' [20].
Normalisation is the work that actors do as they engage with some ensemble of activities (that may include new or changed ways of thinking, acting, and organizing) and by which means it becomes routinely part of already existing, socially patterned, knowledge and practices [20, p. 540].The basic tenet of the theory is that when organisations are confronted with a change they must find ways of accommodating that change.The theory, therefore, aims to develop an understanding of the process by which an information system is implemented, accepted and used.
In particular, the theory is concerned with three issues: 1) Implementation: These are the processes of bringing a practice or practices into action.
2) Embedding: The processes through which practices become or do not become are routinely part of the everyday work of individuals and groups.
3) Integration: The processes by which practices are not only reproduced but are sustained in organisational processes.
This means that first, work practices are normalized when people work either individually or collectively to endorse them.Secondly, the processes involved in enacting a practice is enhanced or inhibited through the operation of some social processes through which human action is expressed.These processes are called generative mechanisms and are coherence, cognitive participation, collective action, reflexive monitoring [18].Third, the production and reproduction of a practice require continuous investment to ensure its sustainability [20].
The four generative processes underpin the three core issues and are discussed below:  Coherence: This is the process of understanding that allows or prevents the use of a practice by participants [17].Coherence involves four sub-components which are differentiation, communal specification, individual specification and internalisation.
 Cognitive participation: This involves anything that allows or prevents users' involvement in a practice [17].It involves the work undertaken to engage the participants who are part of the new intervention.It is this engagement that will position the actors for collective action [21].Cognitive participation covers four sub-components.These are initiation, enrollment, legitimation and activation.www.ijacsa.thesai.org Collective action: This involves the work performed by individual or groups [17].Achieving this goal may include resistance, subversion or reinvention from the users [21].The components of this mechanism are interactional workability, relational integration, skillset workability and contextual integration.
 Reflexive monitoring: This promotes or inhibits users' understanding of the effects of a practice [17].The collective action and the outcomes should be continuously evaluated, both formally and informally, by participants engaged in the implementation processes [20].The components of reflexive action are; systemisation, communal appraisal, individual appraisal and reconfiguration.
According to [20], NPT is a theory of action and is different from other theories because it seeks to explain how innovations are becoming routine in an organisation by focusing on individual and collective learning.In the literature robust social science theories already explain individual differences in attitudes to new technologies and practices (e.g.Theory of Planned Behaviour) [22], the flow of innovations through social networks (e.g.Diffusion of Innovations Theory) [23], reciprocal interactions between people and artefacts (e.g.Actor Network Theory) [24].NPT, therefore, explains phenomena not well covered by existing theories.NPT may shed light on why some IS normalise while others do not [18] and as [25] puts it, NPT offers a coherent framework of propositions that may provide useful insights in the way systems become normalised within organisations.The sub-components mentioned above and they mean in this study is expatiated in Table 1 below.[26].Appraisal work that may lead to attempts to redefine procedures

IV. RESEARCH METHODOLOGY
The study uses an interpretive case study method [27], [28].Following the interpretive tradition [29] means that the philosophical assumptions underlying this study are a subjective epistemology and the ontological belief that reality is socially constructed.These assumptions supported the researchers to understand the behaviour of students and staff in social and organisational contexts by assuming that as they interact they create subjective meaning through their interactions [30].
Multiple data collection methods [27] through documents interviews and participant observation were used.Interviews were the primary data source because it is through this that the researchers' best accessed the interpretations of participants' actions and the events taking place [31].Valuable insight was also being gained from the analysis of research conducted by the AQAU.These secondary data supported the preparation for interviews and helped the researcher to learn about the key stakeholders, technical details and other organisational issues.However, the access to documents and staff of the AQAU as well as students and lecturers was duly guided by the appropriate procedures for gaining access [32] such as endorsements and familiarity with some interviewees [39].Purposive sampling was used to identify interviewees [33].The number of interviewees was not limited to a particular number but continued until a number was arrived at heuristically.This meant the researcher only stopped interviewing when it was realised that nothing new was being gathered from the interviews.Semi-structured interviews were used because of its flexibility to explore emerging themes during the interview.Each interview lasted between 20 to 25 minutes.In all 19 participants were interviewed initially but 2 follow-up interviews was conducted.However, two key participants were contacted so many times over the period of the research.The interviews were digitally recorded and transcribed using NVivo 10 as the data management tool.www.ijacsa.thesai.orgThere is a thin line separating the data collection and the data analysis.This is because the two belong to an iterative process and the results can help guide the other.The data collected was analysed using hermeneutics.This is because hermeneutics is consistent with the interpretive qualitative study.This analysis technique was used because it is consistent with the type of data that was collected.Hermeneutics is primarily concerned with making meaning of textual data by providing a set of concepts to help a researcher interpret and understand the meaning of the text or multiple texts.Hermeneutics is the view that the understanding of a research phenomenon is derived through an iterative process between the understanding of the interdependent meaning of the parts and the whole [27].The process of data analysis involves a number of stages involving the stages of familiarisation, identification of a thematic framework, indexing and interpretation [34], [35].The first stage involved the familiarisation with the data.This was done by going through the interview transcripts several times.This enabled me to fill in the gaps I had missed either through the transcription or during the interview.Some facts were also cross-checked with my interview notes.This was followed by identifying the themes and concepts from the transcripts and putting this into a framework.The framework relied on the four main constructs of the NPT (Coherence, Cognitive participation, Collective Action and Reflexive monitoring) as the main codes and the related subconstructs to guide the sub-themes.

V. CASE STUDY DESCRIPTION
The fieldwork for this study was conducted at UNID in close collaboration with its Academic Quality Assurance Unit.The AQAU oversees the standards of academic work in the university by supporting developing world-class human resources and capabilities to meet national development needs and global challenges through quality teaching, learning, research and knowledge dissemination.The AQAU has several mandates one of which is to conduct student evaluation of courses and lecturers.The evaluations are conducted on every course and teaching staff every semester.The first researcher was attached to the unit for a period of six months as part of a PhD experiential learning.The majority of the first researcher's time was spent on the campus interacting with students, lecturers and administrative staff of the AQAU.

VI. ONLINE EVALUATION OF LECTURERS
The University conducted a paper-based evaluation of courses and lecturers for a long time until 2014 when it was stopped.During this time the University ensured that all departments had a procedure in place for dealing with student evaluation of courses and that this was clearly communicated to students.All students taking a course completed a questionnaire that was prepared by the AQAU and administered by the department through the lecturer.The questionnaire had two main sections; an objective portion where students selected the most suitable option and a subjective or written portion for comments from the students.Students were required to complete both sections of the evaluation form.
The online evaluation was developed by the AQAU in conjunction with a UNID IT Department (ITD).Whilst AQAU handled the administrative aspect of determining the content of the evaluation form and how the data will be analysed, ITD was involved in the technical aspect of developing the web page and making sure that this was up and running during the period of the evaluation.When evaluations are completed, ITD extracts the data and hands it over to AQAU for analysis.However, any feedback received by AQAU from the use of the system is communicated to ITD for improvement in subsequent evaluations.
The online evaluation of lecturers was provided through the University's website.An active link is provided about three weeks to the end of the semester at the homepage of the University website.A click on the link directs students to a login page where a student number and pin is required.After logging in the student is presented with options to choose his/her college first and then department.After this, the courses the student has registered for the semester, the name of the lecturer, the academic year and the semester are populated in a drop down list.After choosing these, the student then proceeded to start the evaluation which was in three main parts; course evaluation, lecturer evaluation and comments and suggestions for improvement.
The systems have evolved from the previously scanable forms.When the online system was first implemented the students were granted access to log into the systems using a security token in order to enable them to conduct the evaluation.When this was implemented, the response rate was quite high but in the subsequent evaluation, it dropped drastically.When the AQAU interacted with some students it was realised that students were sceptical about conducting the evaluation because of fear of getting their identification (IDs) tied to the evaluation.
In the following semester, the feedback of the students was taken into consideration and the token and log in approach were abandoned.An open link was then provided at the homepage of the University website where the students could just visit and start filling out the form without having to log in with the IDs.However, this approach was saddled with issues such as multiple evaluations by students without being noticed.Even a lecturer who feared that he may be evaluated negatively could visit the page and evaluate himself multiple times in order to raise his/her score.To ensure that students did not feel that their identification is tied to the evaluation, AQAU and ITD organised a demonstration session with a cross section of students who were very conversant in the way this type of technology works.This was to allay the fears of the students.Other problems were student complaints that they could not find their courses in the online system.Some students complained of missing course codes, course names and lecturer names.Also, it was reported that the system did not provide avenues for lecturers who had co-taught a course to be evaluated individually.

VII. ANALYSIS OF FINDINGS AND DISCUSSION
In terms of coherence, the participants had a shared understanding of why the online course evaluation was introduced and what are the expected benefits (communal www.ijacsa.thesai.orgspecification).This was much clearer with the staff at the academic quality unit.However, this shared understanding is not being translated into use.The unit responsible for this exercise is investing efforts to make the system better but this is coupled with declining use by students.There are no organised fora to discuss declining student use with students or section of them.There is a clear gap in these shared understanding being translated into shared involvement and participation by students and use by the majority of the students.
The three categories of participants demonstrated different levels of understanding of the aim, objectives and expected benefits of course and lecturer evaluation generally and the online evaluation in particular (differentiation).The participants understood that the online evaluation differed from the paper-based course and lecturer evaluation.This understanding was reflected in the attitude of students towards conducting the evaluation online as opposed to the former paper-based one; an attitude that reflected both scepticism and interest.This is probably because the online evaluation is new and this may be the semblance of the acceptance and use of a newly introduced technology [36], [37].
Overall, several initiatives have been taking place to ensure that students, lecturers and administrative staff understand their specific tasks/responsibilities in the online evaluation process (individual specification).These are evidenced by emails sent to academic staff to remind them to alert students to evaluate courses and flyers posted at student's hostels and lecture halls.Despite these efforts, some difficulties are still being encountered to ensure a full uptake of the online evaluation by students.Some students still claim they have not heard of the online evaluation before whilst others have exhibited the lack of seriousness to conducting this exercise.
All the participants understood the importance of the online course and lecturer evaluation exercise (internalisation).The unit emphasised the need for the University have migrated from the paper-based to the manual, citing the cost cutting reasons, faster processing times and the need to ensure that students get the level of quality of teaching they expect when they come to the University.The reason that the new online evaluation systems would result in improved evaluation of course and lecturers was seen as an important reason to go online.However, the online evaluation required a new approach to making it work.One student participant felt unsure of how the University wanted to achieve the level of quality they needed if student participation is getting lower since and the evaluation processes goes on unnoticed by many students.However, some students think this exercise is a platform they can use to get back at their lecturers who they feel have delivered poorly.Even though the benefits and the importance of the online course and lecturer evaluation are popular among staff, low student patronage could still occur in future evaluations.Students may have to use the new systems as if the paper-based evaluation method never existed.This is because the continuous comparison of the old and the new continues to draw the line indicating how the current method is not being patronised.
In terms of cognitive participation, the determination of the University to build and sustain the new online evaluation process is high (initiation).Equally, the other participants are aware of how the process can be driven forward despite the barriers that are being encountered.The University has been using several approaches to ensure that students are properly (re)organised to engage in the evaluation at the end of every semester (enrolment).This is directed at ensuring that courses/lecturers are collectively evaluated by the students.Though the use of the online evaluations has not reached the level compared to the paper-based one, there is the need to improve communication between all participants in order to ensure that there is common footing with regards to the idea behind the migration from the paper-based to the online evaluation.Student's participants were fully knowledgeable of the need to evaluate their courses and lecturers though majority are still not doing it.The university understands this and is cognisance of the fact that the evaluation process cannot be mandatory for students.A lecturer indicated that: Through my lecturers, When I start my lectures for the semester I tell the students that they will have to evaluate the course and the lecturer at the end of the semester.During my last lecture, I remind the students to go online and do it if I don't forget.This is evidence of how lecturers are trying to get students to be involved in the exercise, however, a comment by a lecturer that: I am not sure lecturers were involved in any way in this new process, at least I never heard of this movement until the end of one semester when the Quality Assurance sent an email to the staff list about how the semester evaluation was being done.
This shows how lecturers were not involved in the migration to the online evaluation.Both lecturers and students indicated that they were not involved in the migration from the paper-based evaluation to the manual one (legitimation).They believed it would have been proper to be involved and that they could have made some critical contributions.Lecturer 1 indicated that: I am not involved much.When it was paper-based we used to support the process by taking the evaluations forms to the last lecturer for student to go through the exercise but now we just have to tell the students in the last lecturer to go online and do it and that's it.
The lecturers and students did not play any part in defining the actions and procedures needed for the online course and lecturer evaluation to work (activation).This was solely decided by the University.The online evaluation has evolved to its current form because of the University's commitment to ensuring that it succeeds.In terms of collective action, the main issue was concerned with all what the participants did in order to involve each other to ensure that the online course and lecturer evaluation is operationalized (interactional workability).To ensure that the new practices of evaluating courses/lecturers online is fully enacted, the University is met with difficulties in fully getting student participation, the lecturers do their best to let students participate in the exercise www.ijacsa.thesai.orgby giving reminders during their lecturers and some of the students want to their colleagues to participate by informing them.However, there is still resistance on the part of students because of the fear of being victimised if they leave negative comments for a lecturer who is under performing.Apart from the lack of fear, involvement by students is sometimes deterred by problems in the online evaluation systems itself as reported by one student that: Well I can't say I completed the process because I needed to evaluate 6 lecturers and I ended evaluating only 1, I didn't even go halfway.This is because either I couldn't find the course code or the lecturers name was not there.
In terms of relational integration, it was clear that the conduct of the online evaluation process is distributed among the participants in the University (skill-set workability).The University does this through flyers to students, and emails to the academic staff.Though the University is aiming at incorporating the online course and lecturer evaluations within the context of the university there is much they can do to let students fully embrace it (contextual integration) though there is a limit to what can be done to ensure that students must do it.
In terms of reflexive monitoring, the participants were able to determine how effective and useful the online course and lecturer was for them (systemisation); along the way, the University to appraise the worth of the online course and lecturer evaluation (communal appraisal).Collectively this can be done by all participants.From the lecturers and students, the general feedback is for the University to intensify its awareness among students.The individual participant's made attempts to appraise the effects of the evaluation process (individual procedures).The general feedback from lecturers and students were that the university needs to improve the awareness of this exercise especially in the student community.Other complaints were that of missing course codes, course names and lecturer names.Also, it was reported that system did not provide avenues for lecturers who had cotaught a course to be evaluated individually.
The online evaluation process is a constantly evolving one and continuously needs to be appraised so that the procedure that can make it a success are redefined (reconfiguration).Among all the participants interviewed the general call is for the university to intensify awareness campaign especially among students to ensure that there is high patronage of the system.The University admits it is working to ensure this.Apart from these, other issues such as the difficulty in locating courses have been identified as a key problem.There were complaints of the possibility of students or lecturers going online to evaluate a course and lecturer multiple times since no log or security is required before the exercise can be done.
The research findings show that the University virtualised its course and lecturer evaluations because of several reasons.The cost of printing, administering and processing the survey results were the most compelling reasons.The time savings in terms of administration and processing of feedback was also key [1].The time savings related to students too because they would have some time in their own time to reflect on their answers before they submit [9].Whilst the issue of instilling objective evaluations in students was cited and that the online evaluations will eliminate the possibility of students feeling intimidated in the presence of their lecturer, some students complained about their anonymity in the online process for fear that their identities can easily be tracked in an online system as compared to the paper-based one they have been used to.
The theoretical foundation (NPT) in this study was partly supported by the data collected.There was the presence of the four generative mechanisms of the NPT suggesting that the implementation may have been completed.This is because it is so in some respects.The virtualisation of the paper-based evaluation is completed.However, feedback at the end of each semester is fed back to improve the next semester evaluation process.This is evidence in another respect that the implementation though completed is an evolving process.In its current form, the reflexive monitoring dimension is currently low an indication that the feedback process needs to be intensified for the current evaluation systems to evolve into a better one and also make it sustainable.The presence of the reflexive monitoring dimension is evidence of the possible sustainability of the new system.
The paper-based evaluation was transformed to the online one through a process of creating an understanding of the system by the students who are the key users.However, this was revealed by the data to be low.Engagement of students and lecturers were absent in major aspects of the migration to online.The collective action of the students to conduct the evaluations was high and the lastly not much feedback is received to improve and make the systems more sustainable.The migration to a virtual course and lecturer evaluation was found to be slow especially the awareness need to ensure a full-scale uptake of the online evaluation when it was first introduced a couple of years ago.This confirms the data that whilst coherence is widespread cognitive participation and collective action among students is low.
The effect of the virtualisation of the course and lecturer evaluations was varied.The low response rate is the key effect of the new systems though the issue of convenience and ease of use for of students to conduct the evaluation was also evidenced in the data.
To improve, sensitisation among the student community is key.However, should be directed at reminders for students to conduct the evaluation instead of sensitisation on the objectives of the exercise or why the systems are being introduced.This is because students are fully aware of the objectives and the reasons why the online systems were introduced.Other efforts to encourage student involvement could be through instituting mechanisms such as vouchers to encourage them to complete the evaluation or a feedback process to understand why students are not fully participating.Mechanism should also be put in place to ensure the full involvement in any changes to the process by deepening stakeholder engagement and activation.

VIII. CONCLUSIONS
The study investigated the normalisation of an online course and lecturer evaluation in an HEI.The implementation www.ijacsa.thesai.org of the online course and lecturer evaluation though complete is an evolving process and this is supported by NPT that transformations such as this do not have a complete end point for the implementations process since the systems continue to evolve through constant feedback and update.The study contributes to both IS and HEI literature as an attempt to offering rich insight into how a newly introduced technology can become the normal way of evaluating courses and lecturers in an HEI in a developing country context.It also offers implications for research and practice.For research, the study enjoins IS scholars to move beyond an examination of migration from physical to virtual platforms per se or the introduction of a new technology as a panacea to its normal use and adoption and explore how new technology become a routine use.This research is, however, limited by its single case study nature in one developing country HEI but the findings provide insight into how NPT can be used to explain the normalisation of a technology use.Another limitation is the small number of participants in the study; however, they represented the whole University's participant in the course and lecturer evaluation process and this small sample provided very rich textual data for the study.Future research can compare the experience of different HEIs in different developing countries in order to account for contextual issues.

(
difference between the manual and the online course and lecturer evaluation Initiation: The participants are working to drive the change forward Interactional workability: The work that participants did with each other to operationalize the online course and lecturer evaluation Systemisation: When students attempt to determine how effective and useful the online course and lecturer was for them and others Communal: specification: Respondent's had a shared understanding of why the online course evaluation was introduced and the expected benefits Enrolments: Participants (re)organise themselves and others in order to contribute to the online course and lecturer evaluation collectively Relational Integration: The knowledge work that participants did to build accountability and maintain confidence in a set of practices and in each other as they use them Communal appraisal: When staff attempt to appraise the worth of the online course and lecturer evaluation Individual Specification: Actions that help students, lecturers and administrative staff understand their specific tasks/responsibilities Legitimation: Participants believing it is right to be involved and that they can make a valid contribution Skill set workability: Describes the distribution and conduct of the practices as they were operationalized in the real world Individual procedures: When staff attempt to appraise the effects of them and the context in which they were set Internalisation: Participants understand the importance of the online course and lecturer evaluation Activation: Participants collectively define the actions and procedures needed to sustain the online course and lecturer evaluation Contextual integration: Refers to the incorporation of the online course and lecturer evaluations within the context of the university Reconfiguration:

TABLE I .
NPT ANALYTICAL FRAME FOR THE IMPLEMENTATION OF THE ONLINE COURSE AND LECTURER EVALUATION.ADAPTED FROM