Efforts to quantify the benefit of social program date back at least to Thomas Hobbes, but in the past 15 years there has been a laudable push to subject public and nonprofit programs to greater scrutiny (see Holley and Carr, 2014). Advances in computing, and in the analysis of “big data” have made it relatively easy to create complex metrics and exciting graphics. But despite these innovations, creating meaningful impact evaluations—especially for early-stage ventures—is still quite challenging.
This is because the most rigorous measures of impact, which assess whether a program is actually changing conditions in the world, typically require expensive and time-consuming longitudinal or randomized controlled studies. The results of a 5- or 10-year impact study will arrive too late for those who are being hit by floods or suffering the effects of broken infrastructure today. This is why we need to be both rigorous and “realistic” about what evaluation can offer social innovators and philanthropists (Ebrahim, 2013).
The good news is that by applying a systematic evaluation framework to this challenge, we can create a hybrid approach that combines outcome evaluation with impact estimates based on publicly available data. Please read our impact evaluation white paper to learn more about our approach.