Improving Assessment Management Using Tools

This paper firstly explains the importance of assessment management, then introduces two assessment tools currently used in the School of Information Technology at Deakin University. A comparison of assignment marking was conducted after collecting test data from three sets of assignments. The importance of providing detailed marking guides and personalized comments is emphasized and future possible extension to the tools is also discussed at the end of this paper.


I. INTRODUCTION
Assessment management plays very important role at all educational levels.Good aassessment management helps educators collect data about students' learning and their performance, as well as informing decisions about classroom instruction and curriculum content based on the collected data to personalize students' learning and maximise the outcomes.
The assessment process may consist of a number of tasks.First, creating assessment material, which can be essays, diagrams, drawings, programs, databases, spreadsheets and so on; then collecting the submissions after students complete the assessment; marking the submissions and recording grades before delivering feedback and results back to students.Students then review their results and if they feel they have been unfairly assessed, they can request a remark and marks adjusted accordingly.This whole process is time consuming.Every assessor has their own customized methods and opinions of how to do the assessment.Also every assessor needs to provide accurate and meaningful evaluation feedback to students, which puts a lot of pressure on them.
To meet the assessment needs of both students and staff, the following factors should be considered [1]:  The criteria for marking must meet the objectives of the course.
 The marking must provide a measure of the learning.
 The marking should provide effective feedback to the student.WebQuiz XP, developed by Smartlite [3], can be used to create online quizzes, tests, assessments and questionnaires.A custom explanation message can be shown if a wrong answer is chosen; users can even set a different message according to the answer given.WebQuiz XP also supports surveys or psychological tests, where questions do not have correct answers; this way, it can be used to collect data and display statistics.

II. BACKGROUND AND RELATED WORKS
Deakin University has been using WebCT Vista [4] as its primary on-line learning environment, called Deakin Studies Online (DSO) for a few years.Recently it is changed to Desire2Learn [5], called CloudDeakin.Both these learning environments are powerful teaching and learning platforms providing discussions, whiteboard, content reuse, performance reporting, on-line quizzes, easy-to-manage gradebook and assignment tools, etc.However, very few backend tools are available for assessment management in the WebCT Vista or Desire2Learn environmenst.They do not meet the many varied assessment needs of education today, such as creating a detailed marking guide, or returning students a result report with detailed comments and statistical information about the whole assignment.
For instance, in the School of Information Technology, students are usually required to submit their assignment attempts in electronic form.CloudDeakin provides dropboxs and a simple rubric editing/marking interface.But the majority of marking is still a manual process.As far as the authors are aware, many academics, especially in the mathematics discipline, simply download assignments from CloudDeakin, print them out, mark on paper and return to students.Alternatively they demand hard-copy submissions.Others may mark via the CloudDeakin provided interface directly, which is all right to provide comments, but it is hard to integrate a detailed marking guide or statistical information in the returned assessment report.Even using the rubric functionality in CloudDeakin has limitations.It is rather basic with only one grading scale accommodated across all marking criteria.Some Computer Assisted Assessment tools support the creation of online quizzes with answersr but do not provide the functionality or flexibility that we expect and need for effective reporting of feedback to students, especially the integration www.ijacsa.thesai.orgwith CloudDeakin.Colleagues in the School of Information Technology, Deakin University have developed two assessment tools, Markers Assistant (MA) [6] and On-line Grades System (OGS) [7].
The objectives of both these tools are to reduce the amount of time spent on administrative tasks associated with marking, increase the range of feedback that can be easily delivered to students, and provide an easy interface to access online submissions as well as deliver marks and feedback to students, thus allowing more time to be spent on working one-to-one with students to achieve good results.

III. ASSESSMENT TOOLS COMPARISON
In the following sections, we briefly introduce these two tools (Markers Assistant and On-line Grades System) and then use SIT104 Introduction to Web Development assignment marking as a case study to compare these assessment tools against the above assessment criteria.Conclusions and recommendations are also given at the end of this paper.SIT104 is a core unit in the Bachelor of Information Technology.Students generally complete this unit in the second half of their first year of study.

A. Markers Assistant
Markers Assistant (MA) [6] is a Windows based application developed to provide a flexible method of managing, assessing and delivering results to small and large numbers of students.It is designed to automate as many components of the marking process as possible, whilst maintaining and improving the assessment feedback to the students within a reasonable timeframe [1].It can:

1) provide automation that retrieves student submissions and presents them via a predetermined application
Individual students are identified by information such as student number or ID and an associated email address if results are to be delivered via email.Students can be found either by identifying them in a drop down menu via there ID or email address.A predetermined application is triggered automatically based on the format of the submission.

2) provide a flexible marking guide GUI that only requires the identification of the criteria and/or assessment of an item's value
The marking guide is displayed in the form of checkboxes and is organised in a hierarchical configuration where high level criteria must be met before other items within that criteria are considered, as shown in Fig. 1.

3) automate the calculation of results within the application
Each time a checkbox is ticked the marks are adjusted accordingly.Comments can be added, edited or deleted against an individual item in the marking guide or as a general comment that relates to the submission as a whole.All comments entered are stored in a general repository and can be reused for a another student if required.

4) provide facilities to deliver student results via email
When the marking is complete, all results can be emailed to the students or exported in a file format that is recognised by WebCT or Desire2Learn.

5) provide facilities to collate final marks summaries
The result report is a text file containing the assessment criteria and the associated mark the student received for each item in the criteria.Any comments relating to the assessment are also included, as shown in Table .I. It also provides result summaries of all assessment performed, which is not possible in traditional methods of assessment.MA enables the following statistics [  frequency This analysis data helps students gain a better understanding of their strengths and weaknesses as well as provide them with an overview of all items within an assessment (Table.II).Example MA marking guide www.ijacsa.thesai.org

B. On-line Grades System
On-line Grades System (OGS) [7] is a Web based application developed to provide an easy way to access hyperlink-based assignments, assessing them and delivering the results within a browser.
It is designed to automate hyperlink access, to mark assignments via a GUI and to send feedback to students purely online.It can: 1) provide automation that retrieves student submitted hyperlinks and presents them in a browser.
Individual students are identified by an associated email address.Students can be selected from an email address selection menu.

2) provide an easy to use marking GUI that only requires the identification of the criterion and assessment value for an item after loading a HTML based marking guide.
Detailed marking guide items are displayed in the form of dropdown lists, as shown in Fig. 2.

3) automate the calculation of results within the application
Each time a dropdown list item is chosen, the mark is adjusted accordingly.Comments can be added, edited or deleted at the bottom of the marking guide page.

4) provide a seperate interface to deliver student results online
When the marking is completed, all results can be viewed via a "lookup result" interface by typing the student ID and email address.The result report is a file stored in the underlying database containing the assessment criteria and the associated mark that the student receives for each item in the criteria.All the comments relating to the assessment are also included, as shown in Fig. 3.

5) enable all data collected to be saved and retrieved via a database
Backup has to be done manually to ensure the marking and comments are saved in the database.

IV. COMPARISON AND CONCLUSION
To compare these two tools, data was collected relating to the first and second assignments in the unit SIT104 Introduction to Web Development during trimester 2, 2012.The first two stages of the second assignment were included in the comparison.The unit SIT104 is delivered to students on three campuses, on-campus at Geelong (in regional Victoria), on-campus at Burwood (suburban campus in Melbourne) and off-campus (ie distance education).Example OGS marking guide Fig. 3.

A. Comparison
For assignment 1, both the MA and OGS were used to mark submissions made by the Geelong on-and off-campus students.The MA is used to provide just detailed comments to students but not the mark; the actual marking that was distributed to students was completed using the OGS.
For assignment 2 stage 1, the MA and OGS were both used for marking all submissions.To maintain consistency, only the OGS results were released to students, together with the comments created in MA.
For assignment 2 stage 2, the OGS is used for marking all student submissions.Comments were made as informative as possible on each submission in OGS.
Student feedback was collected via email query, in-class and after-class discussion.
By comparing these three different marking methods and feedback from students, the following similarities and differences are found:

B. Conclusion
The marking guide identifies the expectations of the assessment and the marks provide a measure of the student's success in meeting the expectations.In addition to the marking guide, it would be better to provide information for each marking item which explains what the expectation of each item is.
It was found that when students are given a correct answer they would not only identity where they went wrong for themselves but also discover what is required or is correct.From this point of view, no matter what kind of assessment tools we use, providing detailed and informative comments also help students improve their learning outcomes.
According to a recommendation of the University's Teaching and Learning Committee [8], feedback on assignments to students should:  Be clearly linked to each published assessment criterion  Assist learning, reward achievement, provide encouragement, explain results and enable students to improve their performance The evidence provided above also suggests that a well written marking guide and informative comments does provide meaningful feedback to students.

V. FUTURE WORK
The above two assessment tools remove much of the complication from assessment by automating many of the tedious tasks, which allows assessors to focus on the assessment itself rather than the associated administrative processes.
Also the marking guides created using the tools are reusable and editable for future assessments.
Other features that will further improve the supporting environments and functionality of the two tools include: For MA:  Extend the system to allow annotated attachments to the results

 Provide database connectivity
For OGS  Provide facilities to create and edit a marking guide within the application  Add statistical output.
By combining the positive features of the two tools, a comprehensive, easy to use marking tool can be developed that not only minimises the tedious tasks of marking from an academic's perspective, but also provides students with comprehensive, detailed feedback that will encourage them to learn from their mistakes.