Advances in computing, and in the analysis of “big data” have made it relatively simple to create complex metrics and exciting graphics. But despite these innovations, creating meaningful impact evaluations—especially for early-stage ventures—is still quite challenging.
This is because the most rigorous measures of impact, which assess whether a program is actually changing conditions in the world, typically require expensive and time-consuming longitudinal studies. The results of a 5- or 10-year impact study will arrive too late for those who are being hit by floods or suffering the effects of broken infrastructure today. This is why, according to a recent review of social impact evaluation approaches in the Harvard Business Review, that we need to be both rigorous and “realistic” about what evaluation can offer.
The good news is that by applying a systematic evaluation framework to this challenge, we can create a hybrid approach that combines outcome evaluation with impact estimates based on publicly available research.
To outline our thinking on this topic, Atma Connect has produced a white paper that estimates the benefits of improving early warnings of floods. In this paper, we analyze in depth the benefits of reducing the damage from floods. Economists at the World Bank estimate that floods caused approximately $6 billion in damages to coastal cities in 2005, and that, driven by climate change, this will increase to at least $60 billion a year by 2050.
Looking at Jakarta and nearby communities, recent floods have caused serious damage and loss of life. For instance, in 2007, extreme floods in Central Java province, which flooded 75% of the capital city of Jakarta, led to the deaths of 54 people, forced 200,000 evacuations, and caused approximately $850 million in damage. In 2013, flood waters killed 47 people in the same area (Davies, 2015). Estimated economic losses and damages from the 2013 floods exceeded $575 million, with the biggest losses suffered by retailers (The World Bank, 2016).
- Assuming that AtmaGo can reduce damage from severe floods by 10% through more effective early warnings and improved functioning of drainage infrastructure, this would equate to $56 million to $85 million in avoided damage per severe storm.
- Assuming that AtmaGo can help 50% of the city’s low income residents avoid damage to their mobile phones (which cost on average between $50 to $90) this would provide a benefit of $10 million to $18.5 million to the city’s poorest residents per flood.
The example above focuses on flooding, but Indonesia, like many middle-income countries, also suffers from other challenges, such as extreme traffic congestion, crime, and inequality in economic development, educational resources and health services. Based on our user interview and research, AtmaGo can provide a range of benefits to poor users that can improve community resilience and help people in developing world cities build better lives. Read our white paper to learn more.