Problem sets

banner 4

Problem sets are a common fixture of the Higher Education classroom. They tend to be lists of exercises that students should find a solution to and are mainly used as learning activities for either home or class work. However, problem sets can also be used as part of the assessment mix. Problem sets can be used in formative assessment as tool for supporting students in their understanding and they can be used in summative assessment, whereby the grades awarded for them form part, or even all, of a student's grade for a course. Because problem sets are used as both learning and assessment tools, it is important to clearly explain to students why they are doing problem sets and what will become of their answers.

Advantages of problem sets

  • Allows for both a broader and deeper testing against the course syllabus.

  • May provide a more complete and rounded evaluation of a students' grasp of the subject.

  • The ongoing use of problem sets allows students to receive regular feedback on their development.

  • Using problem sets as summative assessments can be considered a move towards continuous assessment, which is more likely to encourage deep approaches to learning.

Challenges of problem sets

  • There needs to be consideration of the appropriate frequency of summative problem sets.

  • Summative problem sets may need to be changed on an annual basis, or printed solutions withheld, to ensure students do not use solutions from previous years.

  • If feedback is only given on completed solutions then students may miss out on valuable learning opportunities.

  • Problem sets are good at offering the solution but the thinking behind the solution can sometime be overlooked.

How students might experience problem sets

Seen as a particular form of coursework, students may be favourable towards the use of problem sets as summative assessment. From a disciplinary perspective, science students are generally both the most favourable towards it as well as the most exposed to it. Continuous assessment may be seen by students to have authenticity, whilst also removing the stress that such formats give rise to. Moreover, continuous assessment formats are widely perceived by students as more effective in facilitating learning. Problem sets, of course, provide students with regular practice and opportunities for feedback that can help their development. By having some, or all, of the problem sets play an additional summative assessment role, there is necessarily a clear and explicit alignment of the formative and summative activities of the class.

Reliability, validity, fairness and inclusivity of problem sets

Different marking and totalling strategies may be used to calculate the summative contribution of homework – it is important to find the most appropriate strategy and to make sure that all teachers and students are aware of it. The validity of assessing using problem sets can be improved by offering clear descriptions and guidance on marking criteria; appropriate feedback, and forums to discuss these issues. It is not uncommon, particularly in quantitative courses, for problem sets to be more difficult than end-of-year examinations (e.g. demanding higher levels of cognitive activity to initiate classroom discussion and motivate future topics). Care should therefore be taken to ensure a balance between these two functions of problem sets.

How to maintain and ensure rigour in problem sets

A very real concern for many departments that employ problem sets in their teaching is the widespread online availability of previous years’ solutions. While these sources may clearly violate copyright laws, efforts to police them have proven futile. Thus, if problem sets are to be used summatively, such 'solution bleed' should be taken into account; perhaps requiring such assessments to be revised each year. To ensure reliability of marking across particular problem sets, clearly defined marking schemes should be created and departmental practices regarding marking and moderation should be followed. The rigour of problem sets can also be enhanced through ensuring that they are set at the right level for the student being assessed.

How to limit possible misconduct in problem sets

Student collaboration cannot be entirely dismissed, and framing it as misconduct may ultimately be counterproductive (discouraging students altogether from studying together outside of the classroom). Ultimately, an appropriate approach will depend on the motivation for using problem sets in a summative fashion. If the sole intention is to directly influence students to do the problems sets, then it should be unsurprising if students engage with them superficially to achieve this purely results-driven goal. In this case, devoting time to problems sets as classwork for grading could be considered (allowing for a degree of invigilation over the whole process). To promote a deeper, and more honest, engagement with the problem sets one could consider incorporating aspects of self-assessment and/or marking into the problem sets to directly underscore the personal relevance of the work to students.

LSE examples

Further resources

Back to Assessment methods