Disclaimer:
Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.
Overview
Prepared by (Name of the experimenter)
Cristhian Parra
On date (Day/Month/Year)
26/8/2024
Current status of experimental activity
Completed
What portfolio does this activity correspond to? If any
Formalization and SMEs promotion
What is the frontier challenge does this activity responds to?
Formalization and SMEs promotion
What is the learning question(from your action learning plan) is this activity related to?
To what extent can the direct interacitons of waste pickerst with household improve trust between them?
Please categorize the type that best identifies this experimental activity:
Quasi Experimental (Analytical, observations, etc)
Which sector are you partnering with for this activity? Please select all that apply
Public Sector, Private Sector, Civil Society/ NGOs
Please list the names of partners mentioned in the previous question:
Coca Cola Company (private), Fundación Moisés Bertoni (Civil Society), Soluciones Ecológicas (Private), MADES (public)
Design
What is the specific learning intent of the activity?
To evaluate the impact of a pilot to implement a differentiated waste collection service that connects an association of informal waste pickers' with actual homes in 5 neighborhoods of Asunción, the capital of Paraguay
What is your hypothesis? IF... THEN....
IF we establish and facilitate interaction between waste pickers and households THEN trust relationships will develop among the parties, which in turn will contribute to increasing the level of engagement in activities that make part of the collective dilemma of recycling ( e.g., waste sorting at the point of origin). This hypothesis is fully in display on the causal chain our Recycling as a collective action dilemma blogpost.
Does the activity use a control group for comparison?
Yes, a different group entirely
How is the intervention assigned to different groups in your experiment?
Non-random assignment
What is the timeline of the experimental activity? (Months/Days)
6 months
What is the unit of analysis of this experimental activity?
Households and Waste Pickers. While random assignment was not possible, we did use random selection of the sample to include in the evaluation (on the households' side).
Please describe the data collection technique proposed
This was a mixed-methods experiment that had two different ways of collecting data and evaluating impact. Impact evaluation on households: on the households’ side, two surveys were implemented. First, a characterization survey on the whole population of households that signed up for the pilot. Second, an impact evaluation survey collected from a random sample of households that were included in the pilot (the treatment group) and a random sample of households that signed up but were excluded because they fell outside the neighborhoods selected for the pilot (the control group). Only the universe of households that signed up was considered to mitigate the auto-selection bias that was unavoidable in this initiative (i.e., both groups are groups already interested in taking part of this type of services). Furthermore, the random selection strategy for the impact survey ensured balance in both groups across variables of income, employment, and gender, collected in the first survey. Econometric analysis including standard t-tests to compare means and the use of multivariate linear regression to explore interaction among variables were used to estimate impact on recycling knowledge, recycling behavior, trust, among other things. Impact evaluation on waste pickers: in-depth interviews, a cultural probe and a participatory evaluation workshop were implemented to understand how the new service impacted the lives of the 5 different waste pickers who were in charge of providing the service.
Results
Was the original hypothesis (If.. then) proven or disproven?
On the household’s side: our hypothesis about trust was proven. Impact of the pilot on the levels of trust between households and waste pickers increased by approximately 37% (p<0.01). Meaningful interactions were also higher, by roughly 35% (p<0.05), in the treatment group than in control. No other variables were impacted on the households’ side: knowledge about recycling was higher in control (but both were already too high, leaving little room for improvement), interactions between households also showed no differences, sustainable management of waste was also approximately the same in both treatment and control. On the waste pickers’ side: qualitative evaluation identified two key positive impacts, (1) the perception that households started treating them better, and (2) gaining pride on their daily work, particularly on the eyes of their families and children. Other results of note: while knowledge about recycling was found to be already high in the population we engaged, this did not translate into practice as qualitative observations recorded high levels of frustration from waste pickers on the quality of the recyclable waste they received, which often included materials that were not valuable. Moreover, logistics and organization were noted as an element to improve, waste pickers faced higher mobility costs and the return was not necessarily better, households noted that more frequencies and flexibility was needed. Overall, the service is still not perceived to be financially sustainable. Our impact evaluation, however, opened the door to second phase were policy implications and specific and concrete ideas we propose, inspired by the results, are being considered to inform the implementation and scaling up of the pilot in the 2023.
Do you have observations about the methodology chosen for the experiment? What would you change?
We feel confident about our methodology, and particularly, about the fact that we were able to incorporate this in an action that was not fully under our control to design. It serves as a showcase of how to integrate learning in action, compromising just enough. Other than evaluating the impact, the mixed methods we applied enabled us to engage allies in the process of learning and helped them buying into our proposals. The cultural probe we tried, on the other hand, did not work as well. It generated few in-context responses, and they were mostly too short to provide enough richness. To work better, more time would be needed for this type of methods to work in this type of studies. We only had 2 weeks of reported experiences. In-depth interviews were added to complement this and provided great amount of insight. A participatory evaluation workshop, bringing together waste pickers and institutional allies that were part of the organization, was key to identify consensus points regarding the next stage of the pilot, which will begin in 2023. This workshop in particular was highly praised and well received by everyone, while at the same time generated rich data. The workshop started with waste pickers and organizations working separately, evaluating the initiative and proposing ways to improve it. Both groups then switch places: waste pickers discussed and identified potential issues with proposals made by organizations, and viceversa. A final deliberation brought everyone into the same room to discuss dissensus proposals and gain awareness about consensus proposals.
Learning
What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?
We know that trust building is in the pathway towards increasing recycling practices in the city. We know that strengthening cooperative practices and associative organization among waste pickers is a needed investment. We know that knowledge does not translate into practice, but also that households consider recycling a collective action dilemma, opening the way to integrating community participation practices to improve the linkage between households and waste picker associations.
What were the main obstacles and challenges you encountered during this activity?
Logistics and organization of the weekly tours. Issues of breakdown and repair of motorcars that had little support from the social enterprise tasked with coordinating implementation. Financial sustainability for waste pickers: the service is not yet profitable for them, representing often a high opportunity cost (e.g., to attend participants demands, they have to resign other valuables they can collect along the way)
Who at UNDP might benefit from the results of this experimental activity? Why?
Asuncion Green City Project, a large project implemented by the Environment Program of our CO, whose goal is to impact
Who outside UNDP might benefit from the results of this experiment? and why?
The Association of Waste Pickers in San Francisco Neighborhood might benefit if a better coordination and cooperation leads to a higher volume of recyclable recovered and sold by them. They also benefit by strengthening their organization. The Coca Cola Company can benefit by significantly contributing to the recovery of the plastic waste they introduce into the system. The city of Asunción is benefitting by having its first differentiated waste collection service.
Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?
Before the final form, our original plan was to implement a survey to measure the impact of a natural experiment already taking place in the city around the installation of the so-called “Eco-points”: recyclable waste containers installed in companies or other places, managed by a Social Enterprise and attended to by a selected waste picker. This was meant to be stage one, later to be followed up by stage 2, experimenting directly with the prototype of a differentiated waste collection service. In between stages, we were continuing our work with the Association of Waste Pickers, investing in activities to strengthen them as an organization. As we designed stage 1, The Coca Cola Company learned of our efforts in mapping routes and experiences of waste pickers, and approach UNDP Asuncion Green City Project to propose a collaboration around this initiative, which was very close to our definition of the stage 2, so we decided to join the effort and skip stage 1.
Comments
Log in to add a comment or reply.