Ensuring impact: using methods and metrics to assess results


October 2015 — It's a no-brainer: If you want to measure business success, look to the numbers. But with the advent of hybridized "philanthrocapitalism" and technologies enabling the measurement and analysis of practically everything, the same standard has become prevalent in the social sector as well. Giving a clear, quantifiable picture of a program's impact helps beneficiaries and funders alike by ensuring that grant money reaches its target, and helping foundations, governments, and NGOs make better funding choices.

This month, URB.im looks at innovations in quantification from cities around the Global South — programs and practices that give social success a much-needed metric and enable funders to be guided by strategy and statistics instead of sentiment. Follow the conversation as it unfolds over the course of the month and share your thoughts in the comments below.

El Proyecto que innova con la agricultura urbana

Caracas, 28 octobre 2015 — Las fundaciones, así como, los organismos internacionales, se encuentran presionados en obtener la data que refleje los resultados de su propuesta. El articulo intenta comprender como proyectos de pequeña escala y sin la necesaria financiamiento, puedan obtener la capacidad técnica de la data necesaria para obtener dicha financiación. Leer más.

چرا توصیههای بانک جهانی در ایپانزده سال بعد: ارزیابی پروژه نواب

Tehran, 26 October 2015 – پروژه نواب بزرگترین پروژه نوسازی شهری در ایران است. علی رغم این تا ۱۵ سال پس از اجرای طرح هیچگونه ارزیابی اجتماعی و اقتصادی از اثرات آن صورت نگرفت. تا اینکه در سال 1388 شرکت مهندسین مشاور طرح و معماری با همکاری شهرداری تهران ارزیابی این طرح را آغاز کرد.ادامه مطلب.

MIS – Integrasi informasi untuk mengukur Dampak Program

Jakarta, 23 October 2015 — PNPM Peduli yang kini memasuki fase kedua telah bermitra dengan sekitar 76 CSO yang bekerja di lebih dari 200 desa. Luasnya jangkauan program ini memerlukan sebuah system yang mampu memonitor perkembangan program tiap mitra. Untuk itu dikembangkanlah Sistem Informasi Manajemen (MIS) sebagai bagian dari monitoring dan evaluasi program Peduli untuk mengukur dampak program. Baca lebih lanjut.

Assessing malaria control programs in Accra

Accra, 22 October 2015 — With this month's forum aimed at assessing impact, this article focuses on an urban malaria control program in Accra. The Ghana Health Services gathers periodic data on malaria cases reported at various hospitals in the country. Malaria is the leading cause of morbidity and mortality in Ghana, accounting for about a third of outpatient cases noted each year. Most studies on malaria prevalence rates have been focused on rural areas, where the prevalence rate of the disease is perceived to be higher than urban areas. See more.

Measuring impact of education reform

Lagos, 21 October 2015 — An important aspect of a program's success lies in the ability to measure impact. For the Lagos Eko Secondary Project, improving educational standards and infrastructure is measured through student performance and management training progress and its application. See more.

El observatorio ambiental de Bogotá

Bogotá, 120 octobre 2015 — La medición de los impactos de los programas y de las políticas públicas es un elemento clave en el proceso democrático. También lo es el diseminar de forma amplia y comprensible los resultados de estas mediciones entre los ciudadanos. El Observatorio Ambiental de Bogotá es un buen ejemplo de como lograr estos dos objetivos mediante el uso de las nuevas tecnologías. Leer más.

PPAM – Pengentasan Kemiskinan berbasis Partisipasi

Surabaya, 19 October 2015 — Program Pengentasan kemiskinan kerap dirancang tanpa partisipasi aktif masyarakat miskin itu sendiri sehingga indicator yang dibuatpun tidak mencerminkan kondisi lokal. Akibatnya banyak program penanggulangan kemiskinan gagal. Pendekatan dengan Participatory Poverty Assessment and Monitoring (PPAM) mencoba pendekatan yang berbeda. Baca lebih lanjut.

A análise de dados na construção de um futuro especial para pessoas com deficiências

Curitiba, 16 outubro 2015 — Estando imerso num fluxo de trabalho intenso e rotina, aprimoramentos e ampliações podem parecer difíceis, mas, muitas vezes, basta um empurrãozinho para que mudanças sensíveis sejam catalisadas. É esse incentivo que a ASID, Associação Para Igualdade das Diferenças, tem dado em Curitiba desde 2010, fornecendo as competências e ferramentas certas – com foco na análise de dados –, para que escolas e centros filantrópicos de atendimento a deficientes consigam sair do piloto automático e evoluir. Leia mais.

City BRT, neighbourhood BRT, and the "Corridors of Freedom": Image and reality in Johannesburg

Johannesburg, 15 October 2015 — Touted as the "Corridors of Freedom" by the City of Johannesburg, the city’s BRT system has the potential to improve mobility and accessibility and to even restructure the ‘apartheid city’. But one recent study by researchers at the University of the Witwatersrand’s urban resilience research programme and the South African Cities Network revealed many contradictions at the neighbourhood level. Private investment, densification and land development activity therefore must be guided, local impact must be of equal importance to city scale planning, and legal, financial, and spatial instruments need to be aligned. See more.

Creative metrics for diverse portfolios

Bangalore, 14 October 2015 — The Grassroots Business Fund’s success story, Bangalore-based Industree Crafts, is an example of integrating a variety of metrics to gain a deep understanding of its investees and how—and how well—they are working. Dynamic tools like the ones GBF uses facilitate quick adjustments that can guide social enterprises toward a more sustainable and impactful future. See more.

Viviendas mejoran la calidad de vida de los habitantes marginados

Mexico D.F., 13 octubre 2015 — La medición de las políticas públicas y proyectos sociales demuestran si la población beneficiaria está mejorando su calidad de vida; J-PAL y Techo demuestran, a través de la evaluación de impacto, que el progreso de las viviendas mejora el bienestar de sus beneficiarios. Leer más.

Measuring informality in Nairobi's Jua Kali

Nairobi, 12 October 2015 — Nairobi's informal sector contributes considerably to the success of Kenya as an innovation and emerging technology powerhouse. Attempting to capture what exactly the 'hustle' is has been the focus of the work undertaken by Tatiana Thieme. Her work to quantify the spaces of informality in which the majority of Nairobian youths currently operate has shed light on the vibrancy and innovation that typifies their work. See more.

Trung tâm Hỗ trợ Phát triển Cộng đồng LIN: Tổ chức Hỗ trợ Đánh giá Đắc lực

Ho Chi Minh City, 9 Tháng Mười 2015 — Trong khi phần lớn các tổ chức phi chính phủ và phi lợi nhuận được tạo ra để giải quyết các vấn đề xã hội, có một số ngày càng đông các tổ chức khác lại được được tạo ra chủ yếu để hỗ trợ số đông kia trong việc thực hiện công tác xã hội của họ một cách tốt hơn. Trung tâm Hỗ trợ Phát triển Cộng đồng LIN tại TP.HCM là một trong những tổ chức như vậy, là nơi hỗ trợ các tổ chức phi lợi nhuận (non-profit orgarnizations, viết tắt là NPOs) địa phương trong một công việc rất cần thiết nhưng đôi khi thường bị lãng quên - đánh giá. Xem thêm.

Gerando recursos para programas que transformam a realidade social no Brasil

Rio de Janeiro, 8 outubro 2015 — Associações independentes são, em geral, movidas pelo trabalho incansável de pessoas que sabem bem como resolver os problemas da área em que atuam, mas precisam de suporte para medir e expandir o impacto de seus projetos. Neste artigo, conheça o trabalho da Brazil Foundation, organização sem fins lucrativos com sede no Rio de Janeiro que se dedica justamente a isso: a fortalecer e ampliar o trabalho de iniciativas sociais sérias e promissoras. Leia mais.

الصندوق الاجتماعي للتنمية: يغير المجتمع ويُثبت تأثيره

Cairo, 7 October 2015 — أثبت الصندوق الاجتماعي للتنمية (SDF) في القاهرة أنه منظمة ناجحة يمكن أن تحقق أثرا اجتماعيا واقتصاديا مباشرا من خلال مختلف المشروعات الصغيرة والمتوسطة والمشاريع المجتمعية. وقد تبين ذلك من خلال البيانات المستخرجة من الصندوق والتي تهدف إلى إظهار كفاءته كمنظمة تعمل على تغيير الشكل الاجتماعي في مصر. اقرأ المزيد عن ذلك

Cali Como Vamos: midiendo el comportamiento del gobierno municipal

Cali, 6 octubre 2015 — La iniciativa Cali Cómo Vamos es un esfuerzo independiente de monitorear el trabajo de la administración municipal. Genera dos informe anuales, uno basado en indicadores y el otro en un sondeo masivo de la opinión pública. Su éxito ha llevado a que sea implantado en 14 ciudades de Colombia. Leer más.

Measuring the effectiveness of education programs in Abuja

Abuja, 5 October 2015 — In addition to programs' internal impact assessments and evaluation measures, organization such as Centre for the Study of the Economies of Africa provide a non-partisan outside perspective. Through independent impact analysis and evaluation of public programs, they offer rigorous evidence-based research aimed at improving the impact and delivery of programs. See more.

Can good data save lives?

Mumbai, 2 October 2015 — Vera Solutions has helped organizations around the world harness technology to improve their data collection and use. The Mumbai-based company has worked with local NGOs, such as the Foundation for Mother and Child Health, to customize systems that make tracking important data easier to track and analyze. The databases have transformed the way social organizations understand their impact and diagnose the best way forward with making change happen. See more.

মাইক্রো ফ্র্যাঞ্চাইজিং : দুগ্ধজাত খাদ্যে বিষক্রিয়া রোধে ক্ষুদ্র অথচ কার্যকর সমাধান

Dhaka, 1 October 2015 — দাতব্যসেবার আধুনিকায়নের সাথে সাথে দাতারা সামাজিক পরিবর্তনে প্রদত্ত সেবার পরিসংখ্যানগত ভূমিকা নির্ণয়ে আগ্রহী হয়ে উঠছেন। কৃষি উৎস একটি মাইক্রো ফ্র্যাঞ্চাইজি ধাঁচের উদ্যোগ এবং এই মডেলের শর্তাবলির সাথে প্রায় খাপে খাপে মিলে যায়। এছাড়াও বিনিয়োগকারীদের সামাজিক খাতে বিনিয়োগের সুফল পরিমাপ করে দেখিয়ে দেয়। এই পরিসংখ্যান অন্যান্য শহর ও নগরে একই ধরনের উদ্যোগ নেয়ার সিদ্ধান্তকে উৎসাহিত ও পরিপূর্ণ করে কারণ উদ্যোগ গ্রহণের পূর্বেই উদ্যোক্তারা কার্যক্রমের সম্ভাব্য ফলাফল জানতে ও পরিমাপ করতে পারে। See more.

 


Join the discussion for this month's topic!

Comments

I have been fascinated with the environmental conservation in Abuja.Hence iwish authorities in my town could do the same.
From Mangochi town,
Malawi southern Africa.

Hello Carlin,
Great piece! I think, as per Vera solutions, data really saves lives, since M&E now is part and parcel of the project management cycle. When numbers and data is correct, we will have a valid baseline information which will lead to valid and credible final measurements that lead to more knowledge on the efficiency of any project which was mentioned as part of your article and component of the Vera Solutions mission as I believe.

joinafric's picture

In each region set up an office that lists the various investment sectors and have the results in each sectors every 3 years .

María Fernanda Carvallo's picture

Jorge, muy interesante el ejemplo de Cali sobre la encuesta y la herramienta del reporte para dar seguimiento a las acciones del municipio. Sería muy interesante identificar como impacta esta información en las decisiones de la ciudadanía ante la elección de la siguiente administración municipal. Definitivamente mientras más información tenga el ciudadano y de seguimiento a los compromisos del municipio, podrá diferir de un buen gobierno y exigir lo que la ciudad requiere.

Hello Hilary!

Great post! Quantifying the contribution of informal labourers will hopefully produce results which can be used as a counter argument to the variety of threats the workers face daily. Highlighting the way that informal labor is viewed is extremely important as more people who benefit from the convenience of the local botanist or service provider will be more inclined to support informal workers. Hope the work is a success.

Carlin Carr's picture

Ashali, you bring up a great point here: that measuring and counting is not just about validating projects and programs but also about bringing visibility to certain issues and groups. The example of informal workers is an interesting one, and I like how you point out that people who use the services of informal workers would be more likely to recognize and acknowledge them if we can better show who they are and how they contribute to cities. Good statistics also really help in moving forward government policy and planning to support the needs of informal workers, because they often don't know about certain worker groups--take domestic workers, for example, who are often inside homes and quite invisible. Without the numbers to really tell the story of the needs of informal workers, it's difficult to get the right policies and practices in place.

Hilaryzainab's picture

Thank you for your comment Ashali. I agree it is really important to begin to capture and quantify their contribution as they are often painted negatively by the media and government. This is especially true with local officials often profit from extorting these same individuals. This places informal business owners in a precarious situation where they must pay the bribe to continue to safely operate, yet at the same time the loss of these monies is both a hit to the business and a prohibitive factor that makes it difficult for them should they desire to begin participating in the formal economy. It is a catch-22 for many and hopefully the ability to demonstrate the real and substantial impact these individuals and business have on the countries GDP will promote a more equitable and egalitarian method for adapting formal systems to meet the real needs and demands of a rapidly growing economy.

Hilaryzainab's picture

Hey Olatawura, I found your article on the Ecko program in Lagos schools very interesting. I would be interested to learn more about the capacity development learning undertaken with the teachers and the impact this has over time on students performance. It is also very encouraging to see infrastructure and technical development funded alongside traditional education/learning objectives. It is often the case that donors are more apt to fund innovative ideas than the hard more static necessities. In my research in Kenya, I have found a strong relationship between the quality and availability of adequate sanitation and hygiene availability and girls enrollment and performance in secondary school. Look forward to seeing the results of this project as the WB funding finishes up and the local government takes accountability for these initiatives.

Andréa Azambuja's picture

Hi María Fernanda,

Thanks for your article. Your piece shows two very important things: how joining forces is indispensable – including looking for help internationally to solve "small" local problems – and how important it is to know the areas and the families that will be (or already are) beneficiaries of a social project, to go to them, to listen to them, to make field research... so you can really implement the work and get effectiveness. It is amazing how much money is invested in social initiatives without the real needs and priorities are even researched. I showed one clear example of this in Brazil in my article on public transportation: http://urb.im/ca1507rjp – it's really sad and hopeless. How many work (and waste of money) could be avoided if we could count on J-PAL's help since the beginning in all social projects of this world... :) I'm glad they're making a difference!

María Fernanda Carvallo's picture

Hi Andrea, definitely meassuring social programs ensures the impact on beneficiaries. In Mexico before 2000 social programs didn't have diagnostic nor evaluation, but nowadays thanks to the international sphere it is more common to invest the monew in programs that really impact life of persons.

Hello Widya, I found your article on Surabaya very interesting. Including the participation of the target population is crucial. "Whatever you do for me but without me, you do against me" said Gandhi. In Medellin, I see many urban projects that seem great, and that aim at reducing poverty. Sometimes though, I am not sure that they answer the major needs of the population. There also seem to be a great participation of the local participation. However, many times it seems to me that what results from the participative process is not translated in governmental decision-making, which makes participation uneffective, or even "faked"... It is great if in the case you mention, it has a real impact on policy making.

Hola Jorge, a mí también me parece muy interesante la red de ciudades como vamos. Fui a la socialización de la encuesta de percepción de Medellín Como Vamos y me alegró ver que vinieron muchas personas de la alcaldía y de las empresas más importantes de la ciudad (el metro, EPM…). Me pareció que sí podía tener un efecto directo sobre el diseño de las políticas públicas. La satisfacción con el Metro bajó, por el caos de las horas pico, y la empresa anunció la compra de nuevos trenes… También en el contexto colombiano, donde la corrupción es muy presente, me parece una herramienta interesante para asegurarse del buen uso del dinero público. Los ciudadanos esperan resultados medibles. En Medellín, el marketing urbano es excelente...demasiado bueno… Esta herramienta permite ver si las políticas públicas tienen un impacto más allá de toda la propaganda…

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.