Disclaimer:
Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.
Overview
Prepared by (Name of the experimenter)
Javier Brolo and María Inés Castañeda
On date (Day/Month/Year)
August 13th, 2021
Current status of experimental activity
Completed
What portfolio does this activity correspond to? If any
Knowledge management
What is the frontier challenge does this activity responds to?
How to base decisions and behaviors on existing knowledge
What is the learning question(from your action learning plan) is this activity related to?
What makes it easier for decisions and behaviors to be informed by existing knowledge rather than starting from scratch?
Please categorize the type that best identifies this experimental activity:
Pre Experimental (trial and error, prototype, a/b testing), Quasi Experimental (Analytical, observations, etc)
Which sector are you partnering with for this activity? Please select all that apply
United Nations agency, Academia
Please list the names of partners mentioned in the previous question:
We partnered with the global
network to receive training on how to implement positive deviances; The
University of Manchester
Design
What is the specific learning intent of the activity?
We wanted to test whether positive
deviance was a viable method to be used with available data and to identify
places with opportunities to map solutions on waste management. Also, we wanted
to learn if this method could provide a way to base decisions on existing
information.
What is your hypothesis? IF... THEN....
IF we use DPPD using available data; THEN we will be able to identify locations with unexpectedly high performance in waste management where we can map solutions. IF we estimate expected access to waste collection services; THEN we can identify positive deviants (places with unexpectedly high performance and potential sources of solutions).
Does the activity use a control group for comparison?
Yes, a different group entirely
How is the intervention assigned to different groups in your experiment?
Non-random assignment
Describe which actions will you take to test your hypothesis:
Data Powered Positive Deviance (DPPD) is a technique that consists of five stages. For this experiment, only the first two stages will be completed. During the first stage, and evaluation was made to determine whether DPPD was suitable for addressing the development challenge at hand. Among other things, this involved verifying the existence of positive deviants, securing access to data and skills to analyze them, and assuring that if any solutions were to be found it would be desirable to promote them. Then, for the second stage, an analysis was made to determine which municipalities were positive deviants. To do so, a performance measurement was determined for comparing the observed results with the estimated results calculated by a predictive model where relevant structural factors were included. For the analysis three data sources were employed: XII National Population Census and VII of Household; the Municipal Ranking; and the distribution of constitutional budget. It is worth mentioning that to make the analysis possible, it was necessary to save challenges such as integrating the different levels of data (person, household, home, and municipality). Also, it was necessary to include different data types (.csv, .geojson, and pdf). Positive deviants were defined as those municipalities where the percentage of people with access to waste collection services was higher than expected. The expected percentage of people with access to waste collection was estimated through a linear regression model that included structural factors.
What is the unit of analysis of this experimental activity?
Municipality (after integrating multiple levels of analysis).
Please describe the data collection technique proposed
Secondary data. Collected from open data sources provided by official institutions.
What is the timeline of the experimental activity? (Months/Days)
About one week.
What is the estimated sample size?
100-999, More than 1,000
What is the total estimated monetary resources needed for this experiment?
Less than 1,000 USD
Quality Check
This activity is relevant to a CPD outcome, This activity offers strong collaboration oportunities, This activity offers a high potential for scaling, This activity has a low risk
Please upload any supporting images or visuals for this experiment.
Please upload any supporting links
What are the estimated non- monetary resources required for this experiment? (time allocation from team, external resources, etc) If any.
We need access to data, software for analysis, and skills to analyze the data.
Results
Was the original hypothesis (If.. then) proven or disproven?
Proven. Using DPPD allowed us to identify municipalities with higher than expected access to waste collection services.
Do you have observations about the methodology chosen for the experiment? What would you change?
This was a quick and dirty exercise. Our model for estimating expected access to waste collection should be revised based on existing literature in order to select the relevant variables. Using secondary data possess challenges on the validity of the interpretations. Implementing this exercise required a considerable amount of knowledge on how to analyze different data types.
From design to results, how long did this activity take? (Time in months)
Less than one month.
What were the actual monetary resources invested in this activity? (Amount in USD)
US$0.00
Does this activity have a follow up or a next stage? Please explain
Yes. Once the positive deviances have been identified, we developed and proposed instruments for mapping solutions in those locations. Some of the questions we have about the positive deviances include: what is the role of tourism? Are there people with outstanding leadership? For how long have current solid waste management practices been taking place? How did such practices emerge? Who is responsible for the management of solid waste? What are the associated costs of collecting waste? Are waste collection practices established by rules? How are these practices transmitted? How are sanctioned those who deviate from the rules? Are there outsiders involved? Are there differences between the management of individual waste and collective waste (e.g., hotel, school, event)?
Is this experiment planned to scale? How? With whom?
Yes. However, it has not been possible to. One of the positive deviants happened to coincide with a location where UNDP has worked, but it has not started a follow up project. The other locations are not within the prioritized areas of UNDP.
Please include any supporting images that could be used to showcase this activity
Please add any supporting links that describe the planning, implementation, results of learning of this activity? For example a tweet, a blog, or a report.
Considering the outcomes of this experimental activity, which of the following best describe what happened after? (Please select all that apply)
This experiment led to partnerships, This experiment led to adoption of new ways of working by our partners
Learning
What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?
We know that DPPD is a viable methodology using available and open access data. We also know which municipalities have the largest positive difference between expected and observed access to waste collection.
What were the main obstacles and challenges you encountered during this activity?
The main obstacles were the skills needed to process and integrate multiple data types from multiple sources. Also, it's not clear if our model for estimating expected access to waste collection services is sound. Yet, the most difficult part is creating interest in the results and using them to scale.
Who at UNDP might benefit from the results of this experimental activity? Why?
The environmental programmatic
unit, and the risk prevention unit find it relevant to understand in which
municipalities there is higher performance in access to services. All UNDP
projects could benefit from incorporating DPPD in their repertoire of methods
to determine the locations to work.
Who outside UNDP might benefit from the results of this experiment? and why?
The ministry of environment and
natural resources, the municipalities, and other public institutions interested
in evidence-based policies. Also, academic institutions and policy-research
organizations that need additional ways to generate knowledge from existing
data.
Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?
It required a bit of iterations in the data processing and analysis techniques. However, it was a single question and result.
What advice would you give someone wanting to replicate this experimental activity?
Look at
documentation on DPPD, identify people with data processing and analysis
skills. Be ready to use messy data. Understand the technical limitations of the
audiences to understand the process. DPPD is a very counterintuitive idea to
traditional projects. Traditional projects seem to operate under a
"hero" mentality, believing they have the answer on how to rescue
people in most need. It is difficult to sell the idea that projects could be
facilitators of peer-to-peer learning at the local level, where some local
solutions exist that work in the context where needs emerge but aren't being
used by everyone yet.
Can this experiment be replicated in another thematic area or other SDGs? If yes, what would need to be considered, if no, why not?
Absolutely. We could use positive deviance to identify unexpectedly high performance in access to services other than waste collection.
How much the "sense" and "explore" phases of the learning cycle influenced/shaped this experiment? In hindsight, what would you have done differently with your fellow Solution Mapper and Explorer?
We received the training together as a LAB, sensing and exploring informed the topic to be prioritized.
What surprised you?
Which municipalities were
identified as positive deviances was very difficult to anticipate. The results
around Lake Atitlan made sense, because it's an area with poverty, but with
beautiful landscapes. Their people live off tourism and have had a lot of
exposure to foreigners and their practices. However, Ipala and Cotzal were
quite a surprise.
11Sustainable cities and communities
13Climate action
17Partnerships for the goals
Comments
Log in to add a comment or reply.