Disclaimer:
Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.
Overview
Prepared by (Name of the experimenter)
Javier Brolo and Juan Pablo Rustrián
On date (Day/Month/Year)
August 31th, 2022
Current status of experimental activity
Completed
What portfolio does this activity correspond to? If any
Co-creation of waste management solutions
What is the frontier challenge does this activity responds to?
How to improve the collaboration between society and public institutions to increase resilience to climate change
What is the learning question(from your action learning plan) is this activity related to?
How to improve waste management
Please categorize the type that best identifies this experimental activity:
Quasi Experimental (Analytical, observations, etc)
Which sector are you partnering with for this activity? Please select all that apply
United Nations agency, Public Sector, Private Sector, Civil Society/ NGOs
Please list the names of partners mentioned in the previous question:
The environmental programmatic unit from the UNDP Office, Local Governments, Municipalities, Ministries and NGOs related to environmental care.
Design
What is the specific learning intent of the activity?
We wanted to learn what extent the proposals of citizens on how to measure (and observe) progress in environmental protection coincide (or not) with the goals established in the Nationally Determined Contribution (NDC) of Guatemala in 2021. Also, it is expected that the analysis will identify opportunities to generate data from citizen science, especially on how to improve waste management.
What is your hypothesis? IF... THEN....
If we use open software to analyze open ended responses in an online survey, then we will be able to inform decisions on how to measure improvements in waste management considering the perspective of citizenship.
Does the activity use a control group for comparison?
No, it does not use a control group
How is the intervention assigned to different groups in your experiment?
Non-random assignment, Other
Describe which actions will you take to test your hypothesis:
First, a data cleanup was performed, removing the format from the database; demographic variables were added; time variables were created; responses were coded dichotomously. Second, the text was processed using the "Quanteda" function library. The format was cleaned up; articles and prepositions were removed; tokens, word matrix and co-occurrence matrix were created. Third, with the co-occurrence matrix, a semantic network analysis (including visualization) was performed. This allows words to occur with high frequency in the same response, and thus associate their meanings. Groups of words associated with certain ways of measuring progress were identified; for example: “monitoring protected areas or green areas”. Fourth, with the most frequent words, a dictionary was created to link these words with each of the NCD sectors. When reviewing the NDC indicators, it became necessary to unify different sectors due to the similarity of words. Fifth, using the dictionary, all responses containing words associated with ways of measuring and each of the NDC sectors were classified. With this, a database was built coding each of the answers according to the sectors to which it is relevant.
What is the unit of analysis of this experimental activity?
Text (answers)
Please describe the data collection technique proposed
The text are the responses to a survey for the Stockholm+50 national consultation.
What is the timeline of the experimental activity? (Months/Days)
1 Week
What is the estimated sample size?
More than 1,000
What is the total estimated monetary resources needed for this experiment?
Less than 1,000 USD
Quality Check
This activity is relevant to a CPD outcome, The hypothesis is clearly stated, This activity offers strong collaboration oportunities, This activity offers a high potential for scaling, This activity has a low risk
Please upload any supporting images or visuals for this experiment.
Please upload any supporting links
What are the estimated non- monetary resources required for this experiment? (time allocation from team, external resources, etc) If any.
We need access to data, software for analysis, and skills to analyze the data.
Results
Was the original hypothesis (If.. then) proven or disproven?
Proven. The use of open software to analyze open ended responses in an online survey could be useful to inform decisions on how to measure improvements in environmental protection related topics such as waste management, considering the perspective of citizenship.
Do you have observations about the methodology chosen for the experiment? What would you change?
Performing this text analysis is relevant because we consider the perspective from the citizenship, in addition using open software, based on an open-ended survey, helps to have better and fast understanding of each answer, avoiding leaving information out that could be relevant for the NDC.
From design to results, how long did this activity take? (Time in months)
About two weeks
What were the actual monetary resources invested in this activity? (Amount in USD)
US$0.00
Does this activity have a follow up or a next stage? Please explain
The idea of using text analysis tools is to improve or make better decisions on environmental protection related topics, such a waste management, so the follow up is to stablish the next steps to make relevant decision.
Is this experiment planned to scale? How? With whom?
Yes, now that we have a way to use data from open ended questions, we are looking to incorporate the methodology for other projects within the office. We will be informing content for a campaign to increase environmental awareness.
Please include any supporting images that could be used to showcase this activity
Please add any supporting links that describe the planning, implementation, results of learning of this activity? For example a tweet, a blog, or a report.
Considering the outcomes of this experimental activity, which of the following best describe what happened after? (Please select all that apply)
This experiment led to partnerships, This experiment led to adoption of new ways of working by our partners
Learning
What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?
It is viable to use open software to analyze texts that come from open-ended surveys, which are usually difficult to analyze manually. In our case, it is relevant to know actions that come from citizens for good waste management practices. Some measurement proposals can give us the idea on how to scale solutions at the national level, such as recycling plants, circular economy, waste accumulation, etc.
What were the main obstacles and challenges you encountered during this activity?
The main obstacles were the skills needed to process and integrate the data from our sources, knowing the tools to analyze the data.
Who at UNDP might benefit from the results of this experimental activity? Why?
All UNDP units or projects could benefit from incorporating open software for text analysis.
Who outside UNDP might benefit from the results of this experiment? and why?
Those who oversee develop capacities to take care of the environment.
Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?
We needed iterations to finalize the code that made the analysis reproducible, and how to make sense of the results.
What advice would you give someone wanting to replicate this experimental activity?
We would recommend having access to an open software (friendly one), having the basic skills to operate it, in addition, integrate a person expert in qualitative data into your analysis team to have a better understanding of the outcomes in the text analysis.
Can this experiment be replicated in another thematic area or other SDGs? If yes, what would need to be considered, if no, why not?
Absolutely. We could use open software to work with surveys related to any topic, the idea relays on the text analysis tool.
How much the "sense" and "explore" phases of the learning cycle influenced/shaped this experiment? In hindsight, what would you have done differently with your fellow Solution Mapper and Explorer?
The sense and explore areas have defined the topics of interest. The results will be of special interest to mapping activities that will gather data from open ended questions, and to inform the content of messages.
What surprised you?
The perspective from citizenship plays a very important role in decision-making. In this case, the survey responses yield information that is important for the actors who wish to advance in the goals established in the NDC.
11Sustainable cities and communities
13Climate action
17Partnerships for the goals
Comments
Log in to add a comment or reply.