An old debate surfaced this week on a discussion group on which I’ve been lurking about evaluating web content and the value of Results-based Management (RBM). I wasn’t surprised to see it’s still regarded by some as scary, however, as a results-junkie, I can attest to the fact that it actually works if applied correctly. Today I’ll describe RBM in practical terms, and link to some examples, and in future posts I will apply it to web site evaluation. (Yah, I know I keep saying I’ll come back to topics and never do, but I will. I’ve just got a lot of topics on my mind these days.)
Sadly, like Common Look & Feel, RBM seems to have become another Treasury Board directive in which the benefits haven’t trickled down to the people who have to apply it in the government, despite the fact that the non-profit sector has been using it for decades to both secure funding and keep focused on activities that produce desired changes.
Results-based management is a set of concepts and tools to measure results in terms of desired impact. By defining the links between our activities and desired impact we want those activities to have on our community, then measuring activities, outputs and outcomes against those goals, we can determine whether or not our activities are effective, and whether or not they have been achieved for a reasonable cost. This is a much more appropriate framework for defining value to the public than most private sector models.
The logic model is a tool that can be used to define those links, and therefore needs to be thought out prior to developing an evaluation framework. There are six elements to a logic model: inputs, activities, outputs, and three levels of outcomes. Resources like time and money (inputs) are required to write web content or develop a training seminar (activities), that eventually become products or services (outputs), like a web site or a training session. But why create web sites or offer training? The three levels of outcomes define what impact you’d like to have through those products or services. You also need to know who exactly you are trying to reach (target audience) so you can set performance goals in real numbers. For communication and marketing products, like a web site, the outcomes desired usually fit into one of two categories – increase awareness or influence / change behaviour. How many people or groups of people are in your target audience? What do you want them to do with the information you deliver through your web site?
By defining all that you do in the format of a logic model, the focus is always on what you’re trying to achieve through your everyday activities. Just as writing down your career goals for the next 5 years helps you focus on achieving them, writing down the desired impact of your business activities can help keep you focused on obtaining results. It also helps you to avoid doing things just because they are expected, whether that be from your funders or from your boss. And most importantly, it creates a structure against which to develop the evaluation framework that will help you measure what activities are working. When you know what works and what doesn’t, you can ‘stay the course’ by continuing to do things that work or ‘correct the course’ to adjust the activities to things that may work.
After defining the logic model for your organization, whether it be a non-profit or a government program or project, it becomes quite easy to develop an evaluation framework. If you don’t know what you are trying to achieve in real terms, and haven’t mapped out how your activities relate to these long term goals, you’ll end up defining indicators (KPIs in RBM-speak) that don’t mean anything. So, when debating indicators of success, it has to be in the context of what you’re trying to achieve today, tomorrow and in 10 years from now.
One more thing I want to point out about the image of the logic model – there are two types of ‘success’ or ‘value’ in Results-based management – efficiency and effectiveness. Effectiveness is the extent to which your activities have had the desired impact. However, in the public and non-profit sectors, efficiency is also very important. Efficiency can be defined as the cost to obtain these results. For web sites, I usually try to determine the cost-per-reach of a specific product (web site, pod cast, blog) then compare each against each other A WELL AS against the ‘conversion rate’ of my long term goals to decide whether it was worth my time or effort. However, I won’t go into that too much now, because Mark Schacter has written an excellent paper on the measuring efficiency which I suggest you read if you’re developing an evaluation framework. Actually, you should also read this article too, it’s much more eloquent than this blog post.