Do you know the phrase ‘The best laid plans of mice and men often go awry’?
This is one of those cases.
The idea was innovative, yet simple. In 2014, the Government of Nigeria’s Subsidy Reinvestment and Empowerment Program (SURE-P) was pilot-testing a conditional cash transfer (CCT) project to promote uptake of antenatal care and facility-based delivery among pregnant women. Pathfinder was supporting SURE-P to integrate mobile money into the project. Building on these ongoing programs, we created Text2Speak—a new approach to solicit real-time feedback from SURE-P mCCT recipients through a two-way SMS system.
Text2Speak was Pathfinder’s innovative approach to facilitate citizen engagement in health service delivery. By empowering patients to provide feedback on service quality, Text2Speak aimed to generate actionable evidence to improve quality of service in public health clinics in rural areas. Historically, weak accountability mechanisms in these areas have contributed to lower quality services, which lead to low use of services, which result in poor health outcomes.
The evidence was there. Multiple studies from other countries have shown that this approach works—in Uganda, the SMS-based platform uReport was recently awarded the global UN-based World Summit Award mobile, because of the role it has played to enable Ugandans to engage and participate in governance. That’s what we were hoping for.
The best laid plans…
Following a transition in government, the SURE-P program was delayed—and ultimately shut down. The mCCT initiative was discontinued before we were able to roll out Text2Speak.
We didn’t give up. We adapted.
We immediately initiated conversations with our funders to re-strategize and revise project objectives. Text2Speak was rolled out in the 18th month of the 24-month project as a complement to Pathfinder’s m4change application—another program already in place. Through Text2Speak, we sent out SMS surveys to 2,889 patients across nine primary care clinics over a 6-month period.
Only 556 patients responded, a 19% response rate. People were simply not using the system. Something wasn’t working.
We didn’t give up. We asked why.
We surveyed users of the system. Their responses revealed their reluctance to use the tool and led us to some important conclusions. Text2Speak was a tech-centric design, informed by the opportunity to utilize innovative technology, but it failed to fully account for context.
Behavior change involves both processes and people. Text2Speak should have engaged patients, providers, and policy makers in a collaborative forum upfront to confirm the need for a more accountable health system and recognition for the role Text2Speek could play. Patients needed to be assured that their voices would be heard and would make a difference; providers needed to be assured that Text2Speak would not be a threat to their jobs; and policy makers needed to be motivated to champion the cause for higher quality services. Absent these assurances at the outset, buy-in into the program was constrained. Collaborative design, with the users, would have increased adoption of Text2Speak.
Perhaps if Text2Speak had been deployed as a voice survey, there would have been a higher response rate. Perhaps if Text2Speak had conducted an extensive publicity campaign, more people would have been aware of the benefits of participation and more willing to participate. Perhaps if performance-based incentives were introduced based on client feedback, health workers would have been more eager to encourage clients to participate in the survey. A better understanding of the ecosystem where Text2Speak was implemented would have increased our chances of success. Experiences like this, by us and others, underscore the importance of ‘user-focused design,’ and led to the development of the Principles for Digital Development.
We won’t give up. We’ll keep improving.
No one likes to talk about failure, especially in a donor-funded industry where impact and results are determinants of continued funding. But reflecting on things that don’t go according to plan is critical to inform future program design. Despite Text2Speak’s wobbly roll-out, it was an instructive experience with important learnings for those involved in digital health programming. In line with the Principles for Digital Development, future projects will recognize users of new technologies as key stakeholders that must be involved in articulating their own challenges and affirming the most effective solutions.
Attending this year’s Digital Health Forum in Maryland? Be sure to check out Ayomipo’s presentation about this case on December 14 from 2:30-3:30.
Ayomipo Edinger is Pathfinder’s program coordinator for IT & digital health in our Nigeria office.