Disclaimer:
Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.
Overview
Prepared by (Name of the experimenter)
AccLabPY
On date (Day/Month/Year)
18/09/2023
Current status of experimental activity
Completed
What portfolio does this activity correspond to? If any
Capacities for Social and Public Innovation
What is the frontier challenge does this activity responds to?
Building innovation capacities
What is the learning question(from your action learning plan) is this activity related to?
The original learning questions of our action plan are: How to implement and evaluate the impact of an innovation pilot project to improve a specific public service? How to implement and evaluate the impact of an innovation pilot project to improve a specific public policy?
Please categorize the type that best identifies this experimental activity:
Pre experimental (trial and error, prototype, a/b testing)
Which sector are you partnering with for this activity? Please select all that apply
Public Sector, Civil Society/ NGOs
Please list the names of partners mentioned in the previous question:
The National Innovation Strategy (ENI) and the National Science and Technology Council (CONACYT) act as organizers of the Innovative Public Program, where the Acceleration Laboratory provides technical support and mentoring. A team formed by officials from theTechnical Unit of the Social Cabinet of the Presidency of the Republic (UTGS) and the Ministry of Children and Adolescents (MINNA) participated in the Public Innovators Program and were the winners, therefore, the pilot of the Immediate Alert and Response System (SARI) they proposed received a fund to be implemented.
Design
What is the specific learning intent of the activity?
It is a system that allows the identification, attention and immediate response to Children and Adolescents (NNA) in situations of violation of their rights on public roads. The system receives case reports from citizens and records georeferenced data in real time. This information is processed by interdisciplinary teams of the Ministry of Children and Adolescents (MINNA) and the Technical Unit of the Social Cabinet (UTGS). This innovation seeks to establish a system of immediate attention to these alerts and to have precise information for the design of public policies for the care and attention of children and adolescents. Does the digitalization of the interaction between citizens and the MINNA contribute to improving the experience of users who report on cases of children and adolescents in vulnerable situations on public roads, optimizing the response time of the state as a guarantor of rights?
What is your hypothesis? IF... THEN....
If the interaction between citizens and the MINNA is promoted through the use of SARI, then the response time to assist the children and adolescents will be optimized and the experiences of the citizens will improve.
Does the activity use a control group for comparison?
No, it does not use a control group
How is the intervention assigned to different groups in your experiment?
Non-random assignment
Describe which actions will you take to test your hypothesis:
The pilot was implemented for 4 weeks, between the months of November and December 2022 in the Capital and Central departments. The Public Innovators program proposes an evaluation phase of the implemented pilots, seeking to collect information on the functionality of the innovative solution, in order to devise mitigation strategies and the adjustments that are necessary before its scaling and definitive implementation within of the public institution where you stay.
The SARI evaluation consisted of 4 phases, which included data collection techniques such as (1) focus groups with external SARI users, (2) semi-structured interviews with internal users (UTGS and MINNA officials), and (3) heuristic evaluations carried out with user experience (UX) experts. The findings that emerged from this evaluation made it possible to have a guide for modifications that improve the experience of SARI users and adapt to their needs and expectations.
What is the unit of analysis of this experimental activity?
The analysis unit was made up of external users of SARI, that is, citizens in general who usually make reports of children and adolescents in vulnerable situations on public roads. Also, officials from the UTGS and the MINNA were taken as the unit of analysis.
The study unit was established in Asunción and the Central Department.
Please describe the data collection technique proposed
The primary source data collection techniques for each specific objective were as follows:
Focus group
At first, citizens were invited to participate in a SARI test phase by entering: www.sari.gov.py. In the system, communications from children and adolescents in vulnerable situations on public roads were recorded. This process generated a database where a record of contact information of users who used the system was obtained.
In a second moment, these users who participated in the test phase were contacted and invited to participate in a focus group where data was collected about their experience in the use of SARI and also in the use of Fono Ayuda.
Interviews
Interviews were conducted with MINNA and UTGS officials with the objective of knowing their perspectives and experiences regarding the differences in the monitoring and approach processes based on alerts and/or communications made via Fono Ayuda versus the approach processes that were implemented. give from the use of SARI.
The guides for both the focus group and the interviews are available at the following link:
Heuristic evaluation
In order to obtain a perspective from experts, a heuristic evaluation was carried out. This type of evaluation is a method of usability analysis without users, carried out by experts, which consists of evaluating the elements of an interface based on a series of principles called heuristics.
What is the timeline of the experimental activity? (Months/Days)
4 months
What is the estimated sample size?
10-49
What is the total estimated monetary resources needed for this experiment?
Between 10,000- and 20,000 USD
Quality Check
This activity is relevant to a CPD outcome, The hypothesis is clearly stated, This activity offers strong collaboration oportunities, This activity offers a high potential for scaling
Please upload any supporting images or visuals for this experiment.
Please upload any supporting links
What are the estimated non- monetary resources required for this experiment? (time allocation from team, external resources, etc) If any.
Approximately 20% of the time of our Head of Experimentation, 20% of the
time of our Head of Solutions Mapping, 20% of the time of our Head of
Exploration, plus 100% of the time of two R&D specialists hired by the
AccLab. Moreover, the lab also counted on the help of a Research Associate, who
devoted over 20% of her time to both the implementation and analysis of the
experiment, and the Comms Manager supporting all the communications activity of
this process.
Results
Was the original hypothesis (If.. then) proven or disproven?
Not proven
Do you have observations about the methodology chosen for the experiment? What would you change?
For the learning objectives, we should establish a better monitoring and evaluation strategy. As the focus has been on the design and implementation of the solution, the evaluation strategies were not well defined, and there was no rigor in collecting data systematically.
From design to results, how long did this activity take? (Time in months)
4 months
What were the actual monetary resources invested in this activity? (Amount in USD)
For evaluation purposes only:
1. Specialists in R&D ~ $10000
4. Comms ~ $1000
5. Other resources: time invested by AccLab
Members (average or 2h-4h/week over the full period of the pilot), and time
spent by officials from MINNA and UTGS for the intervention.
Does this activity have a follow up or a next stage? Please explain
No
Is this experiment planned to scale? How? With whom?
The Ministry of Childhood and Adolescense has adopted the tool and it is currently maintaining it after we succesfully transferred all the technical resources into their server. There are plans to scale this system at a regional level.
Please include any supporting images that could be used to showcase this activity
Please add any supporting links that describe the planning, implementation, results of learning of this activity? For example a tweet, a blog, or a report.
Considering the outcomes of this experimental activity, which of the following best describe what happened after? (Please select all that apply)
This experiment led to adoption of new ways of working by our partners
Learning
What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?
We learned that the overload of the Fono Ayuda service makes it difficult to provide immediate attention to cases of violation of the rights of children and adolescents, which is why there is a very urgent need to have tools like SARI to accelerate these processes.
What were the main obstacles and challenges you encountered during this activity?
The main obstacles were related to the limited time for the implementation of the pilot. This meant that testing time by internal and external users was limited, and therefore, generated a short margin of time to analyze the potential for adoption of this tool. Also, a difficulty we encountered is that citizens expressed fear for their safety when making reports (for example, having their cell phones stolen), which is a barrier that must be addressed in future iterations.
Who at UNDP might benefit from the results of this experimental activity? Why?
The Inclusive Development portfolio of the country office could benefit, since this tool speeds up the response of institutions to cases of children and adolescents in vulnerable situations on public roads, guaranteeing their safety and encouraging that these institutions do not leave no one behind.
Who outside UNDP might benefit from the results of this experiment? and why?
The main beneficiaries are children and adolescents in situations of violation of their rights since they will be able to be assisted immediately. On the other hand, citizens will also benefit from being able to interact with the government more directly to intervene in these cases. Finally, MINNA also benefits as an institution since SARI allows its performance to be more efficient.
Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?
No
What advice would you give someone wanting to replicate this experimental activity?
The main recommendation is linked to considering a longer period of time for the user test. In addition,
complement all actions with a good communication campaign that disseminates the existence of the platform and its use, with the aim of achieving greater adoption by internal and external users.
Can this experiment be replicated in another thematic area or other SDGs? If yes, what would need to be considered, if no, why not?
Yes, a platform that connects citizens with the government and allows citizens to report irregular cases to speed up the response of institutions can be useful in cases of gender violence, environmental alerts, disaster alerts, among others.
How much the "sense" and "explore" phases of the learning cycle influenced/shaped this experiment? In hindsight, what would you have done differently with your fellow Solution Mapper and Explorer?
The Public Innovators program incorporates into its scheme several sense-making methodologies, mapping and exploration of solutions and allies, as well as the development of hypotheses that guided the design of the innovation pilots. Therefore, this intervention would not have been possible without the tools and techniques of these two phases.
What surprised you?
We were surprised that citizen users rated very positively that SARI allows them to follow up on the case they reported, indicating high levels of commitment from people who use this type of platforms. That is to say, they are really very interested in knowing the outcome of their report and whether the institutions took action on the matter.
3Good health and well-being
5Gender equality
16Peace, justice and strong institutions
