How to measure impact

Evidencing and reporting the benefits of your research

Evidencing impact involves systematic data collection to demonstrate the outcomes and benefits of a project. Gathering such evidence is dependent on knowing what outcomes and benefits to monitor.

The first step for evidencing impact is, therefore, to make a plan outlining an intervention’s pathway to impact. With a suitable plan in place, qualitative and quantitative indicators can be used to collect evidence throughout the project’s lifecycle, linking the elements of the pathway together to show that an intervention has had a meaningful impact.

Jump to: Pathways to impact, Theory of change, Logic model, Indicators of impact, Examples

Pathways to impact

A “Pathway to impact” refers to the strategies and activities that researchers plan and implement to ensure their work generates meaningful and measurable benefits beyond academia. To devise, map and monitor potential pathways to impact, a theory of change and logic model can be utilised. Both are visual tools for structuring and communicating thinking around how an intervention or project will deliver change, but a theory of change is more speculative and broad to guide early-stage planning and a logic model focuses on mapping and monitoring the operational aspects of a project.

Theory of change

A theory of change creates an illustration of why a desired change is expected to happen within the wider context. The theory of change is built from three ‘spheres’:

The sphere of interest

The broader context and the ultimate goals of the intervention. It includes the long-term changes and impacts that the project aims to achieve. Questions to consider when identifying elements for this sphere are:

  • What does success look like?
  • Which field, theme or region would be affected?
  • What sorts of changes do you envisage?
  • If the intervention were not to take place what would happen or not happen?

The sphere of control

The elements that the project team can directly manage and control. It covers the specific activities, outputs, and immediate outcomes that are within the project’s direct influence. Questions to consider when identifying elements for this sphere are:

  • What funding or resources are available?
  • Who will conduct the work?
  • What institutional tools or support are available?
  • What skills or expertise are at your disposal?
  • What low hanging fruit or easy wins are you capable of achieving?

The sphere of influence

The factors and stakeholders that the project can influence but not directly control. It includes the intermediate outcomes and the broader environment that can affect the project’s success. Questions to consider when identifying elements for this sphere are:

  • Which external people or organisations needs to be involved for the intervention to have the greatest chance of succeeding?
  • Who stands to gain most from the project? How might the be positively affected?
  • What external factors, barriers or opportunities could hinder progress or be leveraged to catalyse success?
  • What are the best ways to engage and interact with stakeholders, collaborators, and audiences?

Logic model

A logic model is a visual representation of the relationships between resources, activities, outputs, and outcomes.

Inputs: Resources that are invested into the project. Examples include funding, staff, equipment, capabilities, etc…

Activities: Actions or work packages carried out during the project. Examples include training session or workshop delivery, conducting field work or research, trials or development, etc…

Outputs: Direct product, deliverable or services resulting from the project. Examples include policy briefs, number of people trained, workshops delivered, published reports, academic publications, patents and licenses, material, or contents, etc…

Outcomes: Short to medium term changes resulting from the project, often defined by a change in status quo (e.g. “strengthened” or “increased”). Examples include improved skills, enhanced capacity, or the development of something “new” (e.g. product, knowledge, contents), etc…

Benefits (Impact): Long-term, significant changes which the project is striving towards. Examples include environmental impact via policy or legislative changes to mitigate against climate change or health & wellbeing impact via improved access to medical care for marginalised individuals or groups.

Indicators of impact

“Indicators of impact” refers to measurable variables or metrics used to assess the progress and effectiveness of an organisation’s activities in achieving its intended impact. Common indicators can be qualitative or quantitative.

Qualitative Indicators are descriptive insights for highlighting positive effects and value of a project. Approaches and examples for capturing this data include:

  • Interviews (Provide detailed, narrative insights into experiences and opinions.)
  • Case studies (Offer in-depth, contextual analysis of specific instances.)
  • Letters of support (Written endorsements (from partner, stakeholder, or beneficiary) highlighting positive influence of intervention.)
  • Event records (Details of contribution, and participation.)
  • Feedback and forms (Qualitative responses and suggestions from stakeholders or participants.)
  • Testimonies (Personal accounts and statements from key individuals.)
  • Meeting notes (Records of discussions and decisions.)
  • Collaborative agreements (Documentation of partnerships or collaborations formed as a result of the research.)
  • Project reports, annual reports or commissioned reports (Descriptions or summary of key findings and endorsements of the work conducted (e.g. Forewords).)
  • Policy (References to government or organizational policies influenced or shaped by the research.)
  • Grant applications and successes (Documentation of partners, collaborations, proposals, funding amounts and successes.)
  • Media coverage (Articles, news reports, and media mentions highlighting the significance and reach of the research, including online comments, reposts, or references.)
  • Implementation evidence (Proof of how the research has been implemented in practice, such as new processes, technologies or programs.)
  • Reviews and evaluations (Evaluations and assessments of service or deliverable.)

Quantitative indicators are statistics and metrics showing measurable changes or improvements resulting from research. Approaches for capturing this data include:

  • Surveys (Provide quantitative data through structured questions.)
  • Performance metrics (Measures of efficiency and effectiveness.)
  • Event records (Data on attendance figures.)
  • Media analytics (Numerical data on reach and engagement.)
  • Aggregated totals, values and amounts (Summarised numerical data.)
  • Reports and publications (Figures from findings or total figures of downloads, readership, or citations.)
  • Grant applications and success rates (Documentation of funding amounts and success rates.)

Examples

Below are some examples to illustrate the pathway and types of evidence used to demonstrate impact:

Pathways to cultural heritage impact

Professor David Stevenson's research on the history of the First World War informed initiatives which commemorated the dead, stimulated interest in the wider history of the war and enhanced understanding of its lesser-known stories and events. The impact from their work was the cumulative outcome from diverse engagement with stakeholders, thorough documentation of a variety of evidence, and a track record of media coverage and public reach. Types of indicators and evidence include:

Stakeholder engagement activities with the Imperial War Museum, the BBC and British Library provided a range of evidence to demonstrate the breadth and depth of Professor Stevenson’s influence: 

  • Academic advisory committee participation for an Imperial War Museum exhibit.

  • Consultation for public learning website, Europeana, 1914-1918, led by the British Library, and contribution of 7 authored articles. 

  • Consultation for the BBC series, "Railways of the Great War" with Michael Portillo and accompanying viewing figures.

  • Educational contents for BBC iWonder series and BBC Bitesize features. 

  • Historical programme notes for Sean O’Casey’s play, The Silver Tassie, revived at the National Theatre in 2014.  

  • LSE International History Department podcast. 

  • Consultation for WWI@LSE, an LSE hosted online 2014 exhibition. 

  • Advisory participation for LSE 2019 public exhibition, “Giving Peace a Chance: from the League of Nations to Greenham Common” (linked to the 2019 LSE Festival).  

Media coverage from mainstream outlets such as the Guardian, Wall Street Journal and Daily Telegraph ensured corroboration of engagement activities: 

  • Reviews of learning website by the Guardian and History.

  • Imperial War Museum exhibition coverage from the Wall Street Journal, the Daily Telegraph and the New York Times.

  • Project Octagon national media coverage by the Guardian.

Public engagement data reinforced the reach of Professor Stevenson’s work: 

  • Online visitor statistics of website from British Library 

  • Imperial War Museum exhibition attendance figures 

  • International History Department podcase download figures and visitor numbers for the LSE Festival public exhibition. 

  • Briefing to 100 participants of Project Octagon, a modern memorial organised by 14-18 Now! and the National Theatre created by Turner prize-winning artist Jeremy Deller. Social media responses to Project Octagon, including trending of hashtag #WeAreHere. 

Statement of support from the head of WW1 Galley Content Team, Imperial War Museum reinforced the significance of Professor Stevenson's involvement in the exhibit.

Awards for "Best Permanent Exhibition" at the 2015 Museum + Heritage awards and Winner of the "Exhibition Design" category of the 2015 Design Week Awards showcased the quality of the display.

Pathways to consumer rights impact

Professor Hugh Collins’s research identified gaps in the remedies available to consumers who were victims of unfair commercial practices or scams. His work led to significant legislative changes that enhanced consumer rights and their ability to seek redress from unscrupulous sales practices, which were costing the UK over £3 billion each year. The impact from their work was the cumulative result of their research outputs and stakeholder engagement, resulting in legislative outcomes. Types of indicators and evidence include:

Reports served as valued evidence for linking the evolution of Professor Collins’ work to the legislative changes that were implemented several years later: 

  • Professor Collins was commissioned by Consumer Focus to write a research paper on how a private right of redress might be enacted and what the benefits of such legislation might be. 

  • Professor Collins’s research was pivotal in the 2009 Consumer Focus report, Waiting to be Heard: Giving Consumers the right of redress over Unfair Commercial Practices.

  • That report was jointly used by the Law Commission of England and the Scottish Law Commission in drawing up their consultation paper, "Consumer Redress for Misrepresentation and Aggressive Practices", which was published as a final report in 2012. 

LSE event which Professor Collins organised and was one of the speakers at, was the only public event concerning the Consultation Paper. This event was attended by: 

  • Civil servants from the Department of Business, Innovation and Skills,

  • Officials involved in the day-to-day enforcement of consumer protection rules,

  • Teams from both Law Commissions. 

Statements endorsing Professor Collins’s research in passing the 2013 Regulations from key figures included: 

  • The Director of International Policy Advocacy

  •  A Law Commission team lawyer 

  • The English Law Commissioner for commercial and common law   

Press release from the Government acknowledged Professor Collins’s 2009 report in their announcement of the legislative changes and details of the new regulations. 

Pathways to climate change impact

LSE’s Grantham Research Institute on climate change and th environment (GRI)’s thinking on sustainable growth and long-term infrastructure investments has been taken up by numerous influential actors. Institutions such as the Organisation for Economic Cooperation and Development (OECD), the World Bank, and the European Commission also make extensive use of the Institute’s research on sustainable growth. Elsewhere, financial institutions, particularly multilevel development banks, including the Asian Infrastructure Investment Bank (AIIB), Inter-American Development Bank (IADB), and New Development Bank (NDB) have all cited outputs from the wider body of GRI research in their guidelines. Furthermore, researchers from the Institute have advised governments in more than 10 countries in Europe, Asia, and Latin America and the Caribbean on climate policy formulation, as well on a climate law for the EU. The impact from their work was generated by the Institute’s successful efforts in creating an attractive new narrative around sustainable growth driving global climate action and providing steers to key stakeholders highlighted above. Types of indicators and evidence include:

Events enabled the GRI to interact with key stakeholders, notably: 

  • Launch event for "Better Growth, Better Climate" report at the United Nations (UN) in New York in September 2014, at an event attended by world leaders and by the UN Secretary-General.   

  • 2015 UN Climate Change Conference in Paris, where Professor Nick Stern was included on a list of the 99 global leaders who created the Paris Agreement. 

Statements corroborating impacts of GRI research on global narrative, and on work of the OECD, IPU, and CCC were provided by: 

  • The Executive Secretary of the UN Framework Convention on Climate Change.

  •  The Secretary General of the OECD. 

  • The Secretary-General of the IPU, the President of the Mary Robinson Foundation - Climate Action.  

  • The Chair of the Parliamentary Committe on Climate Change.  

Quote from Prime Minister David Cameron referenced GRI’s climate legislation statistics at the Paris Summit to remind parties that they were already acting on climate change. 

Letters of support from key stakeholders including: 

  • Thank you letters from the New Zealand Minister for Climate Change and the Chair of the Environmental Committee (Parliament) acknowledge GRI’s input into the legal process. 

  • Letter of appreciation from the European Climate Foundation on GRI’s contribution to the campaign for an EU-wide climate law. 

Government reports showcase GRI’s involvement in shaping New Zealand’s national climate change policy: 

  • New Zealand Productivity Commission report, “Low-emissions Economy”, extensively cites GRI work. 

  • Input acknowledged in the Office of the Parliamentary Commissioner for the Environment’s March 2018 report, “Zero Carbon Act for New Zealand”.

Guidance on finance for sustainable infrastructre from major financial institutions references GRI’s research. These institutions include: 

  • The Asian Infrastructure Investment Bank (AIIB), (2017), “Energy Sector Strategy: Sustainable Energy for Asia”. 

  • Inter-American Development Bank (IADB), (2018), “What Is Sustainable Infrastructure? A Framework to Guide Sustainability Across the Project Cycle. 

  • New Development Bank (NDB), “General Strategy 2017-2021”. 

UK government engagement with officials in relevant departments - chiefly HM Treasury, BEIS, and the Department for Environment, Food and Rural Affairs - including via responses to parliamentary enquiries: 

Pathways to patient experience impact

Research led by Dr Sandra Sequiera testing the effectiveness of a scheduling system for managing patients’ antenatal care visits in southern Mozambique reduced average patient waiting times by 100 minutes and contributed to a 16% increase in the number of women receiving the WHO-recommended minimum 4 antenatal care visits. Patients attested to the benefits of the scheduling system in reducing logistical barriers to attending antenatal care services and improving overall household wellbeing. The intervention has since been rolled out across a further 46 antenatal care units and extended to 40 HIV units in public clinics across four provinces in southern Mozambique, covering more than 217,000 patients, alongside a wider programme of capacity-building at the clinic level. The impact from their work was the result of reduced waiting time which improved patient wellbeing, benefited healthcare workers and provided an evidence base for increased investment in health services and the inclusion of the intervention as a national priority for the Mozambique ministry of health. Types of indicators and evidence include:

Participant survey data was used to quantify and qualify the positive changes from the waiting time intervention: 

  • Waiting time for antenatal care was measured by patient-reported survey data for 6,918 women collected by the research team. 

  • Patient and practitioner testimonies from 38 interviews were collected at three clinics in southern Mozambique.  

  • Figures on number of visits, obtained from administrative data contained in facility registers. 

Statements from government representatives re-affirmed the adoption of the scheduling system as a national priority: 

  • Supporting statement from Director-General, National Institute of Health, Mozambique. 

  • Supporting statement from Deputy Director-General, National Institute of Health, Mozambique. 

  • Supporting statement from the Coordinator of the Health Policies and Systems Program, National Institute of Health, Ministry of Health, Mozambique.

Media coverage from various sources including Jornal de Noite and O País as well as government reports.

Guidelines from Ministry of Health issued to all district authorities further reinforces Dr Sequiera’s work’s reach. 

Testimonial from WHO competition committee commending Dr Sequiera’s intervention for both its " strong potential to directly impact future work to reduce wait times and improve flow of care in the high-burden HIV clinic setting and other areas" and the involvement of high-level policymakers in the project team.

WHO Competition award given to research team for this project, acknowledges the significance and wider potential of the intervention.