Participation in decision making and supporting public policy [Experiment #4]

Innovation methods

Data sources

Scaling

Cost

SDG(s)

Sustainable Development Goal(s)

11Sustainable cities and communities

13Climate action

17Partnerships for the goals

Disclaimer:

Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.

Overview

Prepared by (Name of the experimenter)

Javier Brolo and María Inés Castañeda

On date (Day/Month/Year)

December 30th, 2021

Current status of experimental activity

Completed

What portfolio does this activity correspond to? If any

Implementation of environmental public policy

What is the frontier challenge does this activity responds to?

How to improve the collaboration between society and public institutions to increase resilience to climate change

What is the learning question(from your action learning plan) is this activity related to?

How to implement environmental public policies?

Please categorize the type that best identifies this experimental activity:

Quasi Experimental (Analytical, observations, etc)

Which sector are you partnering with for this activity? Please select all that apply

Public Sector, Civil Society/ NGOs, Academia

Please list the names of partners mentioned in the previous question:

Municipality of Guatemala; Ministry of Environment and Natural Resources; ASIES; Vanderbilt University

Design

What is the specific learning intent of the activity?

We want to understand whether people who have closer proximity to decision making are more supportive of environmental policies. This is because it is not clear if greater exposure to the challenges in decision making make people more critical or more understanding of their limitations.

What is your hypothesis? IF... THEN....

IF people participate in decision making spaces; THEN, they will have greater support for environmental policies.

Does the activity use a control group for comparison?

Yes, a different group entirely

How is the intervention assigned to different groups in your experiment?

Non-random assignment

Describe which actions will you take to test your hypothesis:

To test the hypothesis, we will use secondary public opinion data provided by LAPOP in 2019 (www.LapopSurveys.org). This has a representative sample. We will test the hypothesis analytically using econometric techniques. The independent variable is whether respondents have attended a municipal meeting in the past 12 months, and weather people have attended meetings for improving the community.

What is the unit of analysis of this experimental activity?

People

Please describe the data collection technique proposed

Secondary public opinions from surveys

What is the timeline of the experimental activity? (Months/Days)

One day

What is the estimated sample size?

More than 1,000

What is the total estimated monetary resources needed for this experiment?

Less than 1,000 USD

Quality Check

This activity is relevant to a CPD outcome, The hypothesis is clearly stated, This activity offers strong collaboration oportunities, This activity offers a high potential for scaling, This activity has a low risk

Please upload any supporting images or visuals for this experiment.

Please upload any supporting links

What are the estimated non- monetary resources required for this experiment? (time allocation from team, external resources, etc) If any.

This is a low-cost experiment. The data and software used were free, with open access. It does require someone with the required data analysis techniques and a capable computer. The analysis must have taken around 5 hours to complete.

Results

Was the original hypothesis (If.. then) proven or disproven?

Partially proven. We did not find evidence that participating in municipal meetings increases support for increased spending for improving prevention of natural disasters, but we did find increased support among those who attend community meetings.

Do you have observations about the methodology chosen for the experiment? What would you change?

There are many limitations in using secondary data. The questions available may not be valid enough to test the hypothesis. Also, a more rigorous econometric method should be used to test the hypothesis using ordinal variables, although we did consider the survey design when estimating the models. Also, analytical techniques try to approach a ceteris paribus approach used in experimentation; yet this does not address issues accounted by random assignment. For example, people who participate in decision making activities self-select themselves into those spaces and may have fundamentally different characteristics that other people and attitudes from before exposure to decision making spaces.

From design to results, how long did this activity take? (Time in months)

Less than one month.

What were the actual monetary resources invested in this activity? (Amount in USD)

US$0.00 in additional resources (internet or electricity at most).

Does this activity have a follow up or a next stage? Please explain

We used these results to define the question for the next learning cycle. Why do people adopt environmental practices?

Is this experiment planned to scale? How? With whom?

It is planning to scale in several ways. Its results will be used for the next cycle. The methodology will be transferred to local governments and the ministry of environment so they can discover what works, instead of only providing them with the results.

Please include any supporting images that could be used to showcase this activity

Please add any supporting links that describe the planning, implementation, results of learning of this activity? For example a tweet, a blog, or a report.

Considering the outcomes of this experimental activity, which of the following best describe what happened after? (Please select all that apply)

This experiment influenced public policy at a national or local level

Learning

What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?

We know that we can't rule out that some forms of participation in decision making spaces can be associated with increased support for environmental policies.

What were the main obstacles and challenges you encountered during this activity?

Communicating the results to create interest, particularly to develop/transfer capacity to implement these methodologies using open data.

Who at UNDP might benefit from the results of this experimental activity? Why?

The environmental programmatic area, because they are interested in drivers of support for environmental policies; but also, all areas that can make use of methods to analyze open data with open-source software.

Who outside UNDP might benefit from the results of this experiment? and why?

The municipalities and ministry of environment, who are interested in drivers of environmental policies, and have incentives to increase participation in decision making spaces; as well as using the methodologies.

Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?

No, there are no added benefits known in additional iterations.

What advice would you give someone wanting to replicate this experimental activity?

Reach out if you'd like to access the code to replicate it. Using R may require a learning curve. These types of analysis are not easy to communicate to people without technical knowledge. One must be aware of the limitations of using secondary data in the validity of results.

Can this experiment be replicated in another thematic area or other SDGs? If yes, what would need to be considered, if no, why not?

Yes, we can replicate this way of testing hypothesis, and it could be useful to test if participation in decision making spaces is associated with support of policies in non-environmental topics.

How much the "sense" and "explore" phases of the learning cycle influenced/shaped this experiment? In hindsight, what would you have done differently with your fellow Solution Mapper and Explorer?

The selection of the hypothesis to test was a direct result from exploration and mapping activities. We prioritize together based on the opportunities to link grassroots solutions and new sources of data.

What surprised you?

It is surprising that people who participate in decision-making spaces aren't more critical.