Online Public Service Portal re-design (CPII & RFF)

Cost

SDG(s)

Sustainable Development Goal(s)

9Industry, innovation and infrastructure

10Reduced innequalities

17Partnerships for the goals

Disclaimer:

Please be aware that the content herein has not been peer reviewed. It consists of personal reflections, insights, and learnings of the contributor(s). It may not be exhaustive, nor does it aim to be authoritative knowledge.

Overview

Prepared by (Name of the experimenter)

On date (Day/Month/Year)

Current status of experimental activity

What portfolio does this activity correspond to? If any

What is the frontier challenge does this activity responds to?

What is the learning question(from your action learning plan) is this activity related to?

Please categorize the type that best identifies this experimental activity:

Which sector are you partnering with for this activity? Please select all that apply

Please list the names of partners mentioned in the previous question:

Design

What is the specific learning intent of the activity?

Provide hands-on experience for government officials to learn how to create and implement an experiment in the digital transformation of public services so that it’s more accessible to vulnerable groups.

What is your hypothesis? IF... THEN....

Hypothesis: If we make streamline the portal so that it’s less busy, more directional, and aligns with accessibility standards, then not only the general public but also vulnerable groups, especially blind and visually impaired people will use the portal more often Hypothesis 1: IF we improve the features of the search section (so that different keywords can produce search results), THEN people will need less time to do the procedure on the platform, more satisfied and the number of users will increase. Hypothesis 2: IF the login function is simpler, less requirements, THEN people will need less time and easier to carry out the procedure, will be more satisfied and the number of users will increase.

Does the activity use a control group for comparison?

Yes, the same group but before the intervention

How is the intervention assigned to different groups in your experiment?

Non-random assignment

Describe which actions will you take to test your hypothesis:

Provide a cohesive and comprehensive digital governance incubation program for government officials that include capacity building workshops, coaching, sharing sessions, hackathons, experiment design and implementation to hand-hold them through the activity.

What is the unit of analysis of this experimental activity?

The before and after length of time that end users spend on completing the same task? Level of satisfaction of users when using the redesigned portal The accessibility of blind and visually impaired users of the portal (i.e. whether their screen reader software can read and understand the codes of the portal)

Please describe the data collection technique proposed

Observation (watching users, counting time of using the website, etc.) Interviews Google analytics

What is the timeline of the experimental activity? (Months/Days)

11/11/2021 - 16/6/2022 User testing at the Hackathon with the Blind association The Hackathon is a rare opportunity for the coders and end users to sit together for observation

What is the estimated sample size?

10-49, 50-99, 100-999

What is the total estimated monetary resources needed for this experiment?

More than 20,000 USD

Quality Check

This activity is relevant to a CPD outcome, The hypothesis is clearly stated, This activity offers strong collaboration oportunities, This activity offers a high potential for scaling

Please upload any supporting links

Hackathon Government Framing (in Vietnamese) - https://docs.google.com/document/d/1WxhNrPJO-OHPHYNb7nFEDo2iH1LFmU_TRFIDUduzDsI/edit

What are the estimated non- monetary resources required for this experiment? (time allocation from team, external resources, etc) If any.

All of team members are involved. We also hired multiple consultants on ethnographic research, UI/UX design and public service administration experts Time allocated for ethnographic research (each interview takes 1 - 2 hours)

Results

Was the original hypothesis (If.. then) proven or disproven?

The experimentation will finish in May 2022. Proven hypotheses Hypothesis 1: IF we improve the features of the search section (so that different keywords can produce search results), THEN people will need less time to do the procedure on the platform, more satisfied and the number of users will increase. Hypothesis 2: IF the login function is simpler, less requirements, THEN people will need less time and easier to carry out the procedure, will be more satisfied and the number of users will increase.

Do you have observations about the methodology chosen for the experiment? What would you change?

The methodology of ethnographic research and internal government hackathon helps to engage the government officials more deeply in empathizing with their beneficiaries, especially vulnerable people.

From design to results, how long did this activity take? (Time in months)

The whole incubation program takes 8 months, however the experiment itself was 4 months

What were the actual monetary resources invested in this activity? (Amount in USD)

No monetary resource from the Lab but from our partner - the Governance & Participation team - in a 100,000 USD project but we don’t know the exact amount spent for the experiment

Does this activity have a follow up or a next stage? Please explain

No monetary resource from the Lab but from our partner - the Governance & Participation team - in a 100,000 USD project but we don’t know the exact amount spent for the experiment

Is this experiment planned to scale? How? With whom?

Similar to question 28. And we will pass on the incubation model to our national partner - the National Innovation Center so that other provinces in the country can also apply.

Please add any supporting links that describe the planning, implementation, results of learning of this activity? For example a tweet, a blog, or a report.

CPII Incubation ideas - https://docs.google.com/presentation/d/1BeTlC8VNPfdOPve5MxD-5bh9vvoJ6Y_n/edit#slide=id.p3 Ethnographic design (in Vietnamese) - https://docs.google.com/presentation/d/1qqmxiv2urBKDvMt3nUD9kIPaCebQ2AicMWNq0VmX7O4/edit#slide=id.p

Learning

What do you know now about the action plan learning question that you did not know before? What were your main learnings during this experiment?

What were the main obstacles and challenges you encountered during this activity?

Who at UNDP might benefit from the results of this experimental activity? Why?

Who outside UNDP might benefit from the results of this experiment? and why?

Did this experiment require iterations? If so, how many and what did you change/adjust along the way? and why?

What advice would you give someone wanting to replicate this experimental activity?

Can this experiment be replicated in another thematic area or other SDGs? If yes, what would need to be considered, if no, why not?

How much the "sense" and "explore" phases of the learning cycle influenced/shaped this experiment? In hindsight, what would you have done differently with your fellow Solution Mapper and Explorer?

What surprised you?

Comments

Experiment

Online Public Service Portal re-design (CPII & RFF)

Posted by Nguyen Tuan Luong

UNDP Vietnam Accelerator Lab