How do we know we are making an actual difference?
Many non-profit organizations are being asked to demonstrate the value and impact of their programs and services. Impact measurement is the new normal for providing credible and useful information in decision-making and proving positive change. Many funders now ask, “is the program effective?” or “why does the program work?” or “how might it be modified for improvement?”. The way to deduce is to measure against objectives.
This trend is called evidence-based decision-making or outcomes measures. It is a form of data collection used in monitoring and evaluation. In layman’s terms it means “how do we know we are making an actual difference”?
Why would a non-profit organization implement this newer practice? Partly, it is funders who demand outcomes measures to justify a return on their investment. It is also a way to ward off complacency, because if your organization continually reflects on the data, you can ascertain how well you’re reaching your objectives – pass or fail. When organizations track their impact, it better demonstrates the benefits to people’s lives after participating in your programs. Previously, it was adequate to measure inputs and outputs, but those numbers don’t measure change. Positive change is the supreme goal!
In order to quantify something, one must know what they want to measure. What is your non-profit hoping to achieve? What are its objectives? An outcome is something that changes as a result of something we are doing. The outcomes chosen should be directly related to the mission and vision of the organization. For example, an outcome for a work training program might be “participants secure full-time employment”.
Outcomes are expressed in short, medium, and longer-term transformations. You are evaluating noticeable differences in the knowledge, skills, behaviours, values, and conditions of a participant as a result of your program. Although, it is not possible to take 100% of the credit for the change due to other contributing factors, it is still a probable conclusion to report that the program made a positive difference to the participant.
How does an organization collect this information? A non-profit should track a combination of statistics and stories. Collect only the information you’ll use and use all the information that you collect. This will make analysis easier and result in more accurate conclusions. A blend of qualitative and quantitative data provides a more balanced interpretation of human change. Quantitative data is numbers that can counted. Qualitative data is narratives and expressed feelings.
Once the desired outcomes are distinguished, then indicators must be created. An indicator is simply a way to measure against your goal. For example, a qualitative short-term indicator of a work training program might be that a participant expresses a change in self-confidence after learning how to create a proper résumé. A quantitative indicator could be the number of job interviews resulting from the proper résumé. Indicators are a sign of progress.
Start with a logic model. Think through everything that goes into a program and everything that comes out as a result of the program. Design or revise your program to achieve the outcomes you have selected. Once the conceptual framework is finalized, then begin collecting data at the short, medium, and longer-term junctures using both qualitative and quantitative data collection methods.
Evaluation helps us to understand if a program is achieving its intended outcomes and more importantly why it does! Having such data is valuable. It helps to demonstrate you’re meeting your objectives, or are in close range of meeting them. Measuring impact shows your organization is adaptive to today’s best practices. It is also very satisfying to produce evidence that your programs actually make a difference.
Monitoring and Evaluation Toolkit CIVICUS
Logic Models William McNamara