Working in teams to design, administrate and mark assessment is widespread in universities. This collaborative approach can reduce the workload for individual colleagues, yet when the work produced by a large cohort of students is marked by many markers, marking variation is likely to occur, i.e. a single piece of work may be marked differently by different markers. The quality of marking can be further affected by having frequent changing markers. Research shows the following protocols can counter this marking variation issue.
Marking Calibration before the marking window
This involves all markers discussing their viewpoints around a particular assessment piece to reach a consensus. This method may require significant effort by all teaching staff. The complexity of the assessment task and the characteristics of markers should be considered when deciding whether this process should be adopted.
Please click on image to enlarge
Marking guidance to support individual markers during the marking window
During the marking window, it is important to remind all colleagues of processes, and any shared resources to support them, and with seasonal markers, the course convener should schedule check-ins where required. The following shared resources are found useful by university teachers:
-
FAQs on the marking criteria
-
A personal marking log file, which encourage markers to be self aware of any personal biases and keep them in check
-
A team channel for quick communication between markers
-
Exemplar essays or model answers
-
A spreadsheet with common phrases for feedback
-
Guidelines on feedback, e.g. how much you should write, what form should it take etc., and/or a couple of ‘good’ examples of feedback from a past cohort.
The table below shows one example of a judgement process when marking a large number of submissions.
title
Stage 1 - pre marking
|
- Get assessment brief, marking guidance and marking criteria familiarised
- Pre-sort submissions in sets according to first scan and perceptions of quality
|
Stage 2 - during marking
|
- Refer to marking criteria
- Check other students' work
- Mark individually and construct the feedback
- Mark in sets and compare their performance
|
Stage 3 - post marking
|
- Final checks and comparisons
|