Marking and moderation

banner 9

There are three approaches to marking summative work at LSE – double-blind marking; sighted double marking, and moderated single marking, involving a second examiner. The default position is double-blind marking but this is not always possible with some assessment methods (such as class participation); therefore, the ‘right’ approach to marking should be based on a reflection of both the assessment method and the assessment condition. Formative marking can be more flexible, and unmoderated single marking (marking by one academic) is possible. Formative work could also involve peer marking (where students mark one another’s work) and self-evaluation (where students are guided through evaluating their own work). Marking and feedback can also occur online using Moodle, and LTI have helpful guidance on e-assessment. Two useful questions that can guide all marking are: How do I know that this is the ‘right’ mark? and How can I prove to my students and external examiner that this is the right mark? The answers to both these questions can be found through having a transparent marking system.

Double-blind marking

First marker marks all assessed work. Second marker marks all work, without seeing the first marker’s grades/comments. Where there are any differences in mark, the two markers discuss and agree the final mark (usually meeting in person, but potentially by email/phone/other).

Pros

  • Reduces ‘random distortions’ which might occur in an individual’s marking, increasing reliability
  • Markers can have slightly different perspectives, leading to a more rounded evaluation
  • Students may understand this to be a fair and reliable form of marking

Cons

  • Agreeing marks may be mainly done by ‘splitting the difference’ which draws marks towards the middle of the available range
  • Writing and grading with an eye to justification may lead to ‘defensive’ grading (and thus lower grades)
  • High overall marking loads for each marker

Sighted double marking

First marker marks all assessed work. Second marker reads all work, with sight of the first marker grades/comments, and evaluates the appropriateness of the first markers’ grades/comments. If the second marker proposes any changes to marks, markers discuss and agree the mark, or second marker changes marks.

Pros

  • Less time-intensive than double-blind marking, while retaining the potential for students to benefit from different perspectives
  • Can be a good way to introduce new markers into an established process
  • Works as a check on the markers as well as the marking

Cons

  • For handwritten exams, sighted marking can require a long turnaround time, as the second marker must wait for all work and marks/comments from the first marker
  • Second marker may be influenced by the marks/comments of the first
  • Pre-existing hierarchies of expertise or experience may impact the process

Moderated single marking

First marker marks all assessed work. Second marker considers (with sight of first marker’s marks/comments) a selection of pieces of work, similar to that sent to an external examiner. A typical sample might include all borderline cases; all work graded ‘fail’; all first class work, and 10% of all other pieces.

Pros

  • On a larger course, moderated single marking can be significantly quicker than double marking
  • A representative sample of work can allow markers to clarify the specific qualities that establish a grade; in this way, the approach need not be less reliable than double marking. The scrutiny of borderline pieces of work can have a positive impact on student’s programme outcomes and degree classification

Cons

  • Some pieces of work only receive one academic’s judgement
  • A sample of pieces of work may not adequately represent the first marker’s work, and thus not allow an evaluation of their overall judgement
  • If the second marker substantially challenges the first marker’s judgement, it may be necessary for another academic to re-do the first marker’s marking

In-class assessments

In-class assessment (presentations, group work etc.) can be marked by any of these methods. For moderated single marking, the seminar or class tutor would be the first marker. The course leader would be the moderator, attending a sample of the work/activity (e.g. 2-3 presentations in a seminar group).

Agreeing assessment questions

Assessment/exam questions are agreed by academic colleagues, and their potential interpretations discussed. This can help clarify expectations for markers. Some departments also agree questions with their external examiner.

Criteria discussion

Marking criteria or standards are agreed in advance by discussion between academic colleagues. As with the agreement of assessment questions, this can help clarify the scope of potential valid answers.

Marking or moderation pre-meeting

Academics meet in advance of marking and discuss 2-3 pieces of work. This can help make explicit the tacit understandings of attainment, and bring markers into closer agreement, reduce the time required later for agreeing marks. It may also widen the kinds of attainment that academics see as valid. (If the marking period is too brief to permit a meeting, academics might wish to use sample work from the preceding year and meet in advance.)

Feedback sheets

Feedback sheets can offer students detail on specific aspects of their work, in a tick-box section or as comments under different headings.

Pros

  • Helps staff and students identify strengths and weaknesses of a piece
  • Saves time (ticks can communicate with more brevity than written comments, particularly for less nuanced areas)
  • If used across different courses, helps students to map performance across different pieces of assessed work

Cons

  • Can be hard to agree relevant categories across courses
  • May not work as well at a high level (for high achievers and in later years) where grade is most likely to be determined by more complex and less obvious factors

Marking and Specific Learning Difficulties

The Equality Act 2010 requires Higher Education Institutions not to discriminate against students with disabilities (including dyslexia) in the following ways:

  • Failing to make a reasonable adjustment where any arrangement or physical feature places a disabled person at a substantial disadvantage;
  • Unjustifiably treating someone less favourably for a reason relating to their disability.

LSE utilises appropriate mechanisms to support students with a Specific Learning Difficulty (SpLD). In some cases, one of these is the provision of a Letter of Notification which advises markers where a student has a SpLD which may have affected their sentence structure, spelling and grammar. Students with a SpLD must have been seen by the Disability and Wellbeing Service, and have provided evidence of the SpLD in order for a Letter of Notification to be attached to their work.

Procedures for the student

1) Student approaches Disability and Wellbeing Service and provides evidence of SpLD that meets requirements. 

2) If the student grants permission, Disability and Wellbeing Service completes Individual Exam Adjustments form and Inclusion Plan for reasonable adjustments across the school.  A Letter of Notification is a common reasonable adjustment for students with a SpLD.

3) Upon student approval of the IP, the student is sent a digital version of the Letter of Notification. It is the student’s responsibility to attach it to their written work e.g. summative coursework. In exams, the Letter of Notification is attached to the candidate’s booklet at the end of the exam by the invigilator. The student is not identified by name or nature of SpLD.

Processing assessed work

The Exams Team within Student Services is responsible for ensuring safe collection from the exam room and handover to the academic department of all exam scripts, including those with a Letter of Notification attached. The Exams Team also ensure that Letter of Notification are attached as appropriate to scripts. The Letter of Notification advises the marker that the student has SpLD conditions which may give rise to characteristics which should not be penalised in themselves, as long as the student's argument and analysis meet the requirements of the marking criteria for the assessment.

Regulation

From a regulatory perspective, a student with a Letter of Notification may appeal on the basis that they believe their work has not been marked fairly. In the event that a student with a Letter of Notification were to appeal their mark on the grounds that they had not been fairly marked, the Assessment Regulations team in Student Services would as first steps conduct an audit to check what steps the department / marker had taken to consider the Letter of Notification when marking, and to ensure that it was attached to the work submitted.

Marking work by students with SpLDs

With work flagged as produced by a student with a SpLD, markers are asked ‘wherever possible’ not to ‘penalise for errors in spelling, syntax, word order, and expression’. Marking criteria relating to structure, writing, grammar, fluency and so forth should therefore be taken out of consideration. The marking judgement should be made on the basis of the remaining criteria. So if (for example) an argument can be discerned despite errors in spelling and grammar, the argument should be evaluated without these errors reducing the grade. However, sometimes these aspects can negatively impact on the other more significant criteria. A piece of work will necessarily get a lower grade if the analysis is not at the required level or if wording is so unclear as to make the piece impossible to interpret.

 

Practical suggestions for marking

 

Before marking

  • Revisit the list of features to look out for on the Letter of Notification.
  • Re-check the Learning Outcomes and marking criteria for the course.

While marking a piece

Focus on attainment of the marking criteria which are not related to structure and presentation. If the writing is hard to follow at a sentence level in some places – does the remainder of the piece indicate what the student means? If the structure is confused – have the points made reached the same level as they would if presented in a connected argument? If so, is there any indication that the student understands them to be associated, or has implicitly made that argument? You may need to read back and forwards more than usual to locate parts of the argument, or examples of evidence. Visually highlighting connected aspects (e.g. with temporary adhesive tags) may help. You may also need to check that your sense of the ‘voice’ of the piece is not affecting your judgement of what the piece has attained. Finally, be aware that the stress of exam conditions exacerbates some features of SpLDs, so these features may be more prominent in exam work.

Between pieces

Some standard marking checks can be particularly helpful:

  • Avoid recency and primacy effect by returning to earlier pieces – revisiting pieces from the start, middle or end of a batch.
  • Avoid halo effect by comparing specific aspects of pieces – e.g. look particularly at the evidence offered in support of an argument, in essays from students with and without SpLDs.

While markers often have model answers in mind, or a sense of what a particular grade ‘looks like’, individual markers should not put in place blanket rules which are not communicated to their fellow markers or to the students. This includes, for instance, not deducting or adding marks under conditions not specified in the criteria, to compensate or penalise students.

When feeding back

It is acceptable to comment on issues related to structure or writing, but it is useful to include the statement that this has not been taken into account when grading, so the student is reassured that the appropriate process has been followed.

 

 

 

 

Back to Toolkit Main page