MCQs

• Can test a wide range of course material in a single examination.
• Are relatively quick to answer.
• Can be combined with other questions types in the same exam.
• Can be easily analysed for reliability and validity.
• Offer the possibility of self-testing and feedback if made available online.
• Are quick to mark.
• Enable feedback on specific questions to be given to whole groups.
• Can be grouped into re-usable questions banks.
• Offer the possibility to pilot the performance of individual questions in advance of the test.

• Writing good MCQs requires practice and training.
• It is possible to guess correct answers.
• It is challenging to write questions that test higher order learning.
• Knowledge can be atomised and de-contextualised.
• Students need practice before taking a summative MCQ exam so that they are being tested on their knowledge of the material and not on their understanding of the question type.
• MCQs should be piloted before being used summatively.
• MCQ question banks need to be monitored to maintain integrity.

Students often find formative MCQs helpful in evaluating their learning because they receive instant marks, and, in the case of online quizzes, there is the potential for instant feedback. With their potential to cover the breadth of the course, MCQ exams can motivate students to learn the entire syllabus rather than simply revising those topics on which they intend to write essays. However, students may perceive MCQs as requiring memorisation rather than more analytical engagement with material. If the aim is to encourage a more nuanced understanding of the course content, questions should be designed that require analysis. For example, students could be presented with a case study followed by MCQs which ask them to make judgements about aspects of the brief or to consider the application of certain techniques or theories to a scenario.

One feature of MCQs is that they can be assessed for validity and reliability in advance of their inclusion in summative assessment. For example, they can be piloted as additional, not-for-credit questions within a larger test or set of practice questions. Most MCQ assessment software will generate statistics about the ‘performance’ of questions and their components. Two standard approaches are classical test construction and latent train analysis (Bull and McKenna, 2004).  Piloting questions with colleagues should also allow the assessment designer to analyse items for ambiguity and the type of learning that is being assessed. For example, are the questions largely requiring recall and comprehension? In answering the questions, are students building upon prior learning or applying concepts and theories?  Are students making judgements about cases or scenarios?  Finally, have the questions been written in an inclusive and accessible manner in terms of language, examples and clarity? By analysing the MCQs in this manner, it is possible for the designer of the assessment to address different levels of learning as well as a distribution of items across the curriculum.

In order to ensure rigour, MCQs should be piloted before inclusion in summative assignments. This could take place via formative assessments and or, in advance, with a group of test-takers whose role is simply to analyse the questions for clarity, difficulty, content and type. It is useful to tag questions and create exams according to what the course team is intending to assess. For example, the examination designer may wish to test both basic knowledge (recall, definitions, etc.) as well as more advanced engagement with the material. For those who have not written MCQs, the process can be quite challenging. It can be helpful to organise a workshop (supported by Eden Centre) or detailed guidance material to aid the design of rigorous and appropriate questions.

For paper-based exams, questions can be randomised so that candidates attempt the same questions but in a different sequence. This can help prevent collusion in the exam hall. If the exam is online, the same principle of mixing the order of questions can be applied. Additionally, if banks of pre-tested, validated questions are available, these can be used to generate unique exams. Because MCQs tend to require mainly surface knowledge of a subject it may help to focus on the assessment conditions as a means of limiting academic misconduct.

EC201 Microeconomic Principles I

This resource from the Eden Digital team demonstrates how to analyse the question components of Moodle-based quizzes: https://lse.atlassian.net/wiki/spaces/MG/pages/49512627/Quiz#Use-MCQ-breakdown-to-analyse-quiz-results

This Eden Digital team blogpost shows how to set up a Moodle online quiz: http://lti.lse.ac.uk/moodleportal/assessment-and-feedback-in-moodle/add-a-moodle-quiz/