|
Day 1-Session 1
|
|
|
10.00-10.30
|
What is policy evaluation?
· Evaluation and the policy process
· Monitoring vs evaluation
· Impact and process evaluation
· Different Meanings of Impact
· Quantitative and qualitative evaluation
Group Q/A and discussion
|
|
10.30-10.40
10.40-11.00
11.00-11.15
|
Evaluating the Problem – Reducing drug-related crime
· Situation Analysis
· What is the nature, magnitude and causes of the problem at hand?
Group Work:
· Problem-Tree analysis – identifying causes, consequences and interacting components. How complex is the complexity of the problem?
Feedback from group work
|
|
11.15-11.30
11.30-11.45
11.45-12.00
|
Identifying Policy Options
· Evaluating existing policy levers – are they working?
· What has worked elsewhere? - Using existing evidence
· Stakeholder involvement
Group Work:
· Identifying policy options for reducing drug-related crime
Feedback from group work
|
|
12.00-13.00
|
Lunch
|
|
Day 1-Session 2
|
|
|
13.00-13.15
13.15-13.40
13.40-14.00
|
Establishing a Theory of Change – Dealing with Complexity
· How is a policy supposed to work?
· Inputs, activities, agencies, outputs, outcomes
· Evaluating policy effectiveness at different levels– individual, social, physical environment, macro-sectorial.
· Testing assumptions using quantitative and qualitative methods/data
Group work:
· Drug courts – building a theory of change for how drug courts are supposed to work
Feedback from group work
|
|
14.00-14.20
14.20-14.40
14.40-15.00
|
Developing a Monitoring and Evaluation Framework
· Selecting indicators
· Developing a monitoring and evaluation matrix
· Data transfer – from Excel to statistical software
· Developing and monitoring delivery trajectories
Group work:
What indicators might be used to monitor and evaluate the effectiveness of drug courts?
Feedback from group work
|
|
|
|
|
Day 2- Session 1
|
|
|
10.00-11.00
11.00-11.30
11.30-12.00
|
Counterfactual Impact Evaluation
· What is ‘counterfactual’ evaluation?
· Interrupted Time Series design
· Matched comparison design (Propensity Score Matching)
· Difference-in-Differences
· Regression Discontinuity Design
· Randomised Controlled Trials
Group Work:
In small groups participants will discuss case studies of each of the counterfactual impact evaluation methods above. These case studies will have been distributed electronically to be read before the training.
Feedback from group work:
What have participants learned about the use of counterfactual impact evaluations for policy evaluation?
|
|
12.00-13.00
|
Lunch
|
|
Day 2-Session 2
|
|
|
13.00-13.30
13.30-13.50
13.50-14.00
|
Process Evaluation
· What is process evaluation?
· Evaluating the fidelity and quality of implementation
· Clarifying causal mechanisms
· identifying contextual factors affecting implementation and outcomes
Group Work:
In small groups participants will discuss case studies of process implementation. Case studies will have been distributed electronically to be read before the training.
|
|
Day 2 Session 2
|
|
|
14.00-14.30
14.30-14.45
14.45-15.00
|
Evidence Synthesis
· Harnessing what is already known
· Different types of research synthesis
· The structure of a systematic review
· Examples of evidence synthesis for policy making
· Sources of sound evidence from systematic reviews
Group work:
In small groups participants will discuss the potential for using evidence synthesis in their policy evaluation work.
Feedback from group work
|
