Name
Charlotte Haberstroh
Department
Government
Overview
A new learning activity on GV4F4 (Policy Advice in Theory and Practice). This is a 2.5hour simulation of a social research organisation preparing a bid for a local transport policy tender. The activity adds a practical component to GV4F4, a module that introduces MSc students to theory of evidence-based policy making, the science-policy nexus, and policy advice and evaluation tools. 35 students were enrolled in the class.
Target audience
Participants were studying towards an MSc in Public Policy and Administration as well as International Migration and Public Policy. Most of them have a career interest in producing policy advice either as part of public administration, or as part of think tanks/NGOs.
Details
The activity puts students into the role of policy evaluation professionals at a fictional research organisation (DemRes) producing evidence for policy in the fictional Evbaland. A local government (City of Drais) has issued an invitation to tender for social research on micromobility (think: e-scooters, docked bicycle schemes). Students adopted roles in one of five groups:
- three research groups (cost-benefit analysis (CBA); randomised controlled trials (RCT); behavioural insights for policy / nudge (BI) who develop proposals
- a management team who decide whether to put in a bid, and if so, which of the groups to assign the task to
- an HR administration team who observe the research groups working, and hand out an Excellence Award to the team with highest quality of analysis and most inclusive decision-making.
The activity has two intended learning outcomes.
1) apply and deepen understanding of policy evaluation research methods and approaches taught in previous sessions
2) experience and reflect on the multiple, and at times conflicting, objectives and constraints that feed into - and thus affect - research for policy advice.
The activity replaces the usual learning activities (readings, 1h lecture, 1.5h seminar) of one out of its 10 weeks towards the end of the module. The core of the simulation is a one-hour meeting, at the end of which the management and HR team present their decisions. Groups have one week to prepare. In the seminar following the simulation, students discuss the lessons drawn from the exercise.
I designed the activity over a period of overall three months, dedicating about 10 full working days to create the scenario and develop its resources:
The Eden Catalyst Fund funded a Research Assistant and professional design of the materials by the LSE design unit. This enhanced the quality of the resources, especially their real-life feel. The activity also benefited from feedback and support from Dr Jenni Carr and the Eden Centre.
When redesigning parts of the module, I wanted to create an opportunity for students to apply their analytical skills to a more practical exercise. I learned more about designing simulation activities by actively participating in SIM-SIG workshops, and about the Eden development fund. This was also an occasion to invest in my professional development in curriculum and learning design.
Impact
The design of the material had a direct impact on the participants’ motivation and engagement in the activity on the day. It helped students’ deep immersion into their roles.
In the activity evaluation survey, an overwhelming majority of respondents agreed that “experiencing a practical social research situation has allowed [them] to deepen [their] understanding of the theoretical social research approaches taught in [the previous weeks” and that it had “allowed [them] to deepen [their] analysis of the practice of social research for public policy.”
Answers to open ended questions included statements such as: “I found it to be very helpful and hope that in the future other courses will begin to incorporate similar exercises in lectures/ seminars”, “I really liked how detailed and how close to real life the exercise. For example, how the call for tender and the organisations were inspired from real life ones.” Students also gave relevant critical comments, such as providing more time for the activity, and rethinking the role of the management team.
The activity’s aim was to deepen students’ understanding of the research approaches for policy studied in the previous weeks. They had definitely reached a deep understanding thereof by the end of the module, as students performed very highly in the formal assessment.
Next steps
I plan to run the activity again in the next academic year, but review it in the light of my evaluation and the feedback I received. I will also adapt it to the LSE Curriculum Shift, and will adapt it to blended learning. I will then consider writing about the intervention for academic publication.
The main advice I would give is that it was worth the effort to take the time to design the activity as well as the resources. I also learned a lot by involving students in the evaluation of the activity, so I recommend leaving plenty of time for evaluating the activity with students at the end. This gives students the possibility to reflect on their experience as well as learn from what others experienced.
Colleagues considering developing their own simulation activity might also be interested in the LSE simulation special interest group LSE SIM-SIG. This is a practice exchange group for colleagues who are planning to use, or are currently using, simulations in the classroom. LSE SIM-SIG will meet once a term, and there is an online space to share resources and support. To join LSE SIM-SIG contact Jenni Carr (j.carr4@lse.ac.uk).