Courses for Core Skills

Policy Evaluation Skills

Overview

This online course will provide an introduction to policy evaluation. It will cover how to evaluate the nature and magnitude of the problem at hand as well as the complexity of the causes, consequences and interacting components of the underlying issues. The course will then cover how to develop and test a theory of change at different levels of analysis and complexity.

The second day of the course will focus on methods of impact and process evaluation. Participants will be asked to read ahead of time case studies of different evaluation approaches that have been used in policy evaluation and discuss them during break-out group sessions.

Day 1-Session 1

 

10.00-10.30

What is policy evaluation?

·   Evaluation and the policy process

·   Monitoring vs evaluation

·   Impact and process evaluation

·   Different Meanings of Impact

·   Quantitative and qualitative evaluation

Group Q/A and discussion  

10.30-10.40

 

 

10.40-11.00

 

11.00-11.15

Evaluating the Problem – Reducing drug-related crime

·   Situation Analysis

·   What is the nature, magnitude and causes of the problem at hand?

Group Work:

·   Problem-Tree analysis – identifying causes, consequences and interacting components. How complex is the complexity of the problem?

Feedback from group work

11.15-11.30

 

 

 

11.30-11.45

 

11.45-12.00

Identifying Policy Options

·    Evaluating existing policy levers – are they working?

·    What has worked elsewhere? -  Using existing evidence

·    Stakeholder involvement

Group Work:

·   Identifying policy options for reducing drug-related crime

Feedback from group work

12.00-13.00

Lunch

Day 1-Session 2

 

13.00-13.15

 

 

 

 

 

13.15-13.40

 

13.40-14.00

Establishing a Theory of Change – Dealing with Complexity

·  How is a policy supposed to work?

·  Inputs, activities, agencies, outputs, outcomes

·    Evaluating policy effectiveness at different levels– individual, social, physical environment, macro-sectorial.

·  Testing assumptions using quantitative and qualitative methods/data

Group work:

· Drug courts – building a theory of change for how drug courts are supposed to work

Feedback from group work

14.00-14.20

 

 

 

 

14.20-14.40

 

14.40-15.00

Developing a Monitoring and Evaluation Framework

·    Selecting indicators

·    Developing a monitoring and evaluation matrix

·    Data transfer – from Excel to statistical software

·    Developing and monitoring delivery trajectories

 

Group work:

What indicators might be used to monitor and evaluate the effectiveness of drug courts?

Feedback from group work

 

 

Day 2- Session 1

 

10.00-11.00

 

 

 

 

 

 

11.00-11.30

 

 

 

11.30-12.00

Counterfactual Impact Evaluation

·  What is ‘counterfactual’ evaluation?

·  Interrupted Time Series design

·  Matched comparison design (Propensity Score Matching)

·  Difference-in-Differences

·  Regression Discontinuity Design

·  Randomised Controlled Trials

Group Work:

In small groups participants will discuss case studies of each of the counterfactual impact evaluation methods above. These case studies will have been distributed electronically to be read before the training.

Feedback from group work:

What have participants learned about the use of counterfactual impact evaluations for policy evaluation?

12.00-13.00

Lunch

Day 2-Session 2

 

13.00-13.30

 

 

 

 

13.30-13.50

 

13.50-14.00

Process Evaluation

·   What is process evaluation?

·   Evaluating the fidelity and quality of implementation

·   Clarifying causal mechanisms

·   identifying contextual factors affecting implementation and outcomes

Group Work:

In small groups participants will discuss case studies of process implementation. Case studies will have been distributed electronically to be read before the training.

Day 2 Session 2

 

14.00-14.30

 

 

 

 

 

14.30-14.45

 

 

14.45-15.00

Evidence Synthesis

·  Harnessing what is already known

·  Different types of research synthesis

·  The structure of a systematic review

·  Examples of evidence synthesis for policy making

·  Sources of sound evidence from systematic reviews

Group work:

In small groups participants will discuss the potential for using evidence synthesis in their policy evaluation work.

Feedback from group work

How our virtual classrooms work

Our Virtual Classroom courses allows you to attend our classroom training without leaving your home or office. We use the ‘Microsoft Teams’ meeting software. Before you attend, please check that you meet the system requirements. If it doesn’t work, try adjusting your settings or contact your IT department to ask about permitting the website.

You are also welcome to use your personal laptop, tablet or even your smart phone. Please just forward our course invite to your personal email, or just copy and paste the link into your browser.

If you encounter any problems, please feel free to get in touch. We can set up a test meeting before the course.

For more information, to book or to register your interest in future course dates

Contact Us