Report date: October 2014 update on longitudinal evaluation 2014 - 2020
Program background: Economic self-sufficiency is a core characteristic of the Unbound program model. Sponsored members and their families meet their basic needs through a combination of their own income-generating activities, access to capital and sponsorship benefits. Staff and families shape the program so that, over time, families rely less on Unbound benefits and more on their own income-generation activities to meet their basic needs. While the direct benefits of the program differ by region and country and even from one sponsored person to another, economic development is a universal desired outcome. Economic security and economic self-sufficiency are other ways that we can look at this complex domain of participants lives that moves beyond just improved income.
We can reasonably say that sponsored families benefit, and their economic situation improves to some degree, during their participation in the program. The monthly influx of benefits or the relief on the family’s budget when school fees are paid means that families are indeed “better off” for the time being. If you or I experienced a 20 or 30 percent increase in our monthly budgets, we would likely have a significantly improved standard of living. But what happens when that influx goes away? Will we be in a better position than before, or will we be back at square one? The answer is strongly connected to how the resources were used and key events that either accelerated or impeded our progress.
Purpose of the evaluation: The purpose of evaluation in this domain is to understand how families’ economic situations change over time with their participation in Unbound. The objective is to answer the basic question: When a family leaves the program, is their economic situation any better than when they began? This gets at the very heart of economic self-sufficiency.
Through this evaluation we hope to better understand how families experience, or do not experience, economic and living situation changes that will put them in a stronger position to move forward in life once they leave the program and the monthly benefit is removed.
Methods and design: Exploring various theories and approaches to measuring economic wellbeing through income, assets and expenditures emphasized not only the complexity of the issue but the insufficiency of any one indicator to provide sufficient, relevant data to make conclusions. The layers of diverse national and local economies around the world, coupled with a non-standard economic threshold for participation in Unbound, bring additional complexity.
In response to the complexity outlined above we developed an evaluation design to address this central purpose and question. The design employs both quantitative and qualitative data collection in a three pronged structure.
Profile – Questionnaire addressing multiple indicators on income, savings, debt and resiliency.
Photos – Series of six photos, including housing structure, living conditions, recent purchases and livelihood started or boosted with support from Unbound.
Progress Out of Poverty Index (PPI) – Country specific score card, developed by Mark Schreiner and the Grameen Foundation, for estimating families’ probability of living below national and international poverty lines.
The strength of this design comes primarily from its ability to triangulate data and build a more complete and deep understanding of complex economic situations.
The evaluation is designed with a longitudinal approach. Families will not be compared to one another across communities, projects or regions and their data will not be generalized. Instead individual families will be compared to themselves over time and within their specific contexts. The same three-pronged data collection method will be employed annually or biennially over the next six years. The data will then be evaluated against previous years’ data to detect change and ultimately determine economic outcomes for families.
To best match the depth of data collection and longitudinal design, each data collection site uses a small random sample of 20 new families who have become sponsored within the past six months. This allows us to identify families who are part of the Unbound program but have not yet experienced significant material benefit. A total of 120 families are part of this six-year evaluation, coming from Unbound programs in Colombia, Guatemala, India, Mexico, Philippines, and Uganda.
In the initial pilot and measurement tool testing, an additional sample of families who were five- or six-year program participants was also created. This allowed us to see if the data collection tools and indicators were reliable and valid to detect change and diverse experiences.
Key findings and conclusions: At this time a general summary of findings would be unhelpful as the purpose of the evaluation is not to compare families to one another or generate averages and trends. Current data serves as a baseline and updates will be shared at the midpoint and conclusions of the longitudinal study.
Melissa Velazquez, Unbound senior evaluation specialist
Becky Spachek, Unbound evaluation specialist