How do you know whether a policy or a program that you’re undertaking is working? What about when the problem that you’re addressing is particularly complex? We know that when we’re driving a car putting your foot on the accelerator makes it go, but we also know that the accelerator isn’t the only thing that causes the car to move – there is a massive range of interrelated and interdependent factors that propel us into motion.
Measuring individual factors becomes especially tricky when addressing wicked policy challenges. Addressing these challenges requires comprehensive community initiatives, but how do we evaluate their success? A recent Workshop on Evaluating Complex Community Initiatives and Policy Interventions hosted by the Evaluation Centre for Complex Health Interventions at St. Michaels Hospital addressed this issue.
The Wellesley Institute’s Bob Gardner presented on how to develop a framework for evaluating wicked policy challenges. Gardner argued that the key is to have clear starting points – if we don’t know what we’re setting out to do how will we know whether we’re making any progress?
Addressing complex social problems means that we need to identify the connections and causal pathways between multiple factors, explain what drives change in these factors and the problem as a whole, identify the levers that will drive change, develop strategies for moving these levers, and set specific short-, medium- and long-term outcomes and identify what is required to achieve them.
But don’t worry – you don’t have to do it alone. The cornerstone for making this happen is to bring together a group of individuals and organizations working on the challenge and to answer the questions together. Chances are you won’t get it right the first time… and that’s fine. The important thing is to set clear starting points and regularly evaluate your success based on your short-, medium and long-term outcomes. It’s better to make a start than to wait for the perfect theory to come along.
Finally, remember that evaluation isn’t the same as auditing. Auditing gives you a fixed result that tells you important things like whether you’ve used your funding well. Evaluation, on the other hand, gives you information about what’s working and what’s not and whether you’re on the right track. Think of evaluation as a learning tool – it helps you to refine your work and gives you an opportunity to try new things.
Keep an eye on the Making Evaluations Matter page for information on future evaluations workshops.