Disclaimer:
Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.
Overview
Prepared by (Name of the experimenter)
Javier Brolo and María Inés Castañeda
On date (Day/Month/Year)
June 4th, 2021
Current status of experimental activity
Completed
What portfolio does this activity correspond to? If any
Waste management
What is the frontier challenge does this activity responds to?
How to improve the collaboration between society and public institutions to increase resilience to climate change
What is the learning question(from your action learning plan) is this activity related to?
How to improve waste management
Please categorize the type that best identifies this experimental activity:
Quasi Experimental (Analytical, observations, etc)
Which sector are you partnering with for this activity? Please select all that apply
United Nations agency, Public Sector
Please list the names of partners mentioned in the previous question:
Communications area at UNDP; Ministry of Environment and Natural Resources; Local Governments
Design
What is the specific learning intent of the activity?
We wanted to
learn if we can use social media to test alternative messages. We have competing
arguments regarding what messages are more appealing and supported by audiences
if messages that appeal to empathy or messages that appeal to personal
responsibility. Messages that appeal to empathy make us aware of the needs of
others, whereas messages that appeal to personal responsibility point out that
one has a personal obligation or should be sanctioned.
What is your hypothesis? IF... THEN....
If people see a message that appeals to empathy rather than personal responsibility, then they will engage more and be more receptive of that message
Does the activity use a control group for comparison?
Yes, a different group entirely
How is the intervention assigned to different groups in your experiment?
Non-random assignment, Other
Describe which actions will you take to test your hypothesis:
We created two messages related to waste classification, one that appeals to empathy and another that appeals to personal responsibility. Then, with the help of the communications team at UNDP, we developed two illustrations that matched each message and scheduled it's publication in the social media pages of UNDP Guatemala.
What is the unit of analysis of this experimental activity?
Individual messages
Please describe the data collection technique proposed
We collected data from the social media platform reports (Facebook, Twitter and Instagram)
What is the timeline of the experimental activity? (Months/Days)
Less than one week
What is the estimated sample size?
2-9
What is the total estimated monetary resources needed for this experiment?
Less than 1,000 USD
Quality Check
This activity is relevant to a CPD outcome, The hypothesis is clearly stated, This activity offers strong collaboration oportunities, This activity offers a high potential for scaling, This activity has a low risk
Please upload any supporting images or visuals for this experiment.
Please upload any supporting links
What are the estimated non- monetary resources required for this experiment? (time allocation from team, external resources, etc) If any.
The costs included the time needed to design and run the experiment. We had help from the communication office designing the visual aids and wording of the messages.
Results
Was the original hypothesis (If.. then) proven or disproven?
Disproven. Contrary to our initial hypothesis, the message that appealed to persona responsibility for waste classification had more interactions than the one that appealed to empathy.
Do you have observations about the methodology chosen for the experiment? What would you change?
The result made
us raise different questions about our methodology. It is possible that the
time of the publications influenced the results, however, the pattern obtained
was consistent throughout the three social media platforms used (Twitter,
Facebook, and Instagram). Also, it might be the case that the message appealing
to empathy was unclear (or maybe it appealed to it by a rational means rather
than an emotional one). Furthermore, it is possible that our assumption about
empathy is wrong and those messages that question directly on responsibility
stand out more. For future experiments like this one, it is important to pay
attention to the time of the publications, the clarity of the messages, and
alternative ways to observe potential or acting. Seeing the number of
interactions may not be the most accurate way to see this. We also had great
limitations to access data beyond reactions, the analysis could be donde further
using a developer account.
From design to results, how long did this activity take? (Time in months)
Less than a month
What were the actual monetary resources invested in this activity? (Amount in USD)
US$0.00 additional costs
Does this activity have a follow up or a next stage? Please explain
Yes. We know have the workflow in place to test alternative messages through the country office social media platforms.
Is this experiment planned to scale? How? With whom?
We want to scale the analytical techniques to use social media data, and the methodology to test alternative messages.
Please include any supporting images that could be used to showcase this activity
Please add any supporting links that describe the planning, implementation, results of learning of this activity? For example a tweet, a blog, or a report.
Considering the outcomes of this experimental activity, which of the following best describe what happened after? (Please select all that apply)
This experiment influenced public policy at a national or local level
Learning
What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?
We now have indications that people give more visibility to messages that appeal to personal responsibility that those that appeal to empathy. Although this is difficult to assess from the methodology used, it is possible that other measures of engagement with show different results, and we cannot differentiate the effect of the image from the wording. We also know that we can compare how receptive audiences are to alternative messages in social media.
What were the main obstacles and challenges you encountered during this activity?
We did not have
a way to randomize the assignment of messages. Although there is no reason to
believe that people who were exposed to one of the messages where
systematically different than the other. We also didn't have much access to
data this stage, to see if users saw both messages, and saw one before the
other. Moreover, it's difficult to create similar messages that only vary in
the strategy to connect with people, there are other components to the message
(such as the image used) that could account for the differences in reactions.
We also don't have a well-developed theory about why people react to a social
media message, which is not necessarily an endorsement or receptivity.
Who at UNDP might benefit from the results of this experimental activity? Why?
Certainly, campaigns endorsed by the Country Office in social media will benefit from testing their messages.
Who outside UNDP might benefit from the results of this experiment? and why?
The Ministry of
Environment and Natural Resources and Local governments are interested in learning how to promote
better waste management practices, but also other civil society organizations
and government agencies could benefit from using social media to test messages.
Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?
We needed iterations to refine the images and the wording of messages to see if their messages were clearly contrasting and understandable.
What advice would you give someone wanting to replicate this experimental activity?
The methodology
can be improved. It's worth trying as a very low-cost experiment and a way to
engage other teams in the country office.
Can this experiment be replicated in another thematic area or other SDGs? If yes, what would need to be considered, if no, why not?
Absolutely, appealing to empathy or to personal responsibility are alternatives to try to promote any SDG beyond climate action and sustainable cities through waste management.
How much the "sense" and "explore" phases of the learning cycle influenced/shaped this experiment? In hindsight, what would you have done differently with your fellow Solution Mapper and Explorer?
Sense has defined the topic we are working on: waste management. Explore has raised interest in the use of social media data as an unusual type of data that has potential.
What surprised you?
We did not
prove the hypothesis, which is good to challenge confirmation biases when
making decisions on how to communicate. Usually, we want to appeal to the human
instinct of caring for others, and the natural response to be helpful when it
doesn't take much effort. However, it seems that social media audiences are
less triggered by empathy, and rather have a stronger sense of acting based on
what's right or should be reprimanded. Of course, much more is needed to
understand. But it's surprising that empathy is not clearly preferred.
11Sustainable cities and communities
13Climate action
17Partnerships for the goals
Comments
Log in to add a comment or reply.