In early 2022, our team at The Agency Fund began investigating ways to support the agency of pregnant women and mothers. How might we help women as they navigate the complex challenges of caring for themselves and their families? To an outsider, the "best" decision a woman can make regarding her birth planning will be unknowable. So a more supportive, flexible, responsive approach seems most appropriate. Following months of research and consultation with experts in maternal and neonatal health, we formed a coalition of partners focused on this question. Specifically, we decided to explore how administrative data and predictive analyses might be used to customize support to women during pregnancy, delivery, and aftercare.
Our initial partners (D-Tree, Living Goods, Reach Digital Health and IDinsight) wanted to transform outcomes for vulnerable and marginalized women across Tanzania, Uganda, and South Africa. So we mobilized the financial resources, technical expertise, and peer collaboration needed to unlock the potential of data to achieve cost-effective impact at massive scale.
Our first step was to interrogate existing program data from the D-Tree, Living Goods, and MomConnect services. Each organization started by identifying the attributes of at-risk client segments. These included:
- Reach: 28% of adolescent mothers registered on the MomConnect perinatal advisory service become pregnant less than six months after live birth of their prior child.1 Such short intervals - known as the interpregnancy interval (IPI) - are almost always unintended and associated with poor health, economic and education outcomes.2 What information or support might adolescents need, to protect themselves from the risks of a pregnancy so soon after giving birth?
- D-Tree: 10% of pregnant clients are identified as having a <20% likelihood of delivering in a health facility.3 Facility delivery, and skilled care during childbirth, is closely associated with reduced mortality and morbidity for both mother and child. What kinds of interactions between women and their health providers might increase the chance of a safe facility-based delivery?
- Living Goods: 10% of children are identified as having a 35% likelihood of defaulting on immunizations. These immunizations are some of the most important interventions to cost-effectively reduce child morbidity and mortality. What beliefs or experiences might focus a family’s attention toward preventive care?
Our partners’ next steps are to design, deploy, and evaluate customized interventions for the identified high-risk client segments. They are also building the data infrastructure needed to conduct ongoing analyses and identify additional impact opportunities. In tandem, we at the Agency Fund are excited to broaden the use of predictive analyses in perinatal advisory services. This could involve new partner outreach, the creation of accessible tools and resources for other non-profits to use, and broader health system strengthening efforts.
Even though we’re just getting started, we are excited to share a few lessons from our initial efforts.
First, there are more steps than people generally expect… Analyzing data is not a magic-wand exercise that instantly amplifies impact. First, you need a stable, well-managed program with sufficient probability of impact. From this foundation, many steps are required to fulfill the promise of data, including:
- Accessing a sufficient volume and frequency of accurate, precise, complete, and relevant data.
- Determining meaningful analyses that are actually feasible!
- Conducting these analyses (with a bunch of data wrangling thrown in for good measure)
- Translating analytical insights into novel program actions or approaches
- Piloting, evaluating and refining those analysis-driven actions
- Mainstreaming actions across a program’s footprint
- Incrementally building the data infrastructure to sustain ongoing evaluation and learning
It took us six months to execute Steps 1-3 for a small subset of potential analyses. In the process, we encountered many obstacles, including impossible analyses (due to unavailable data) and analyses with minimal insight (due to small samples). But we also generated some important insights worthy of further action. We have only scratched the surface of our data’s potential, but we note that fully realizing this potential will require years, not months or weeks.
Second, this is not rocket science. Data focused efforts can be obscured in all kinds of space-aged jargon like “machine learning,” “random forests,” “AUC”, etc. However, our first stage was simply to identify clients with the greatest needs and risks. Such segmentation routinely occurs every day– when done by frontline workers, clinicians, and program managers. Fancy analytics can improve upon everyday prioritization and triage– either by segmenting faster and better than human experts, or by performing segmentations out of reach of human experts (by examining data patterns too deep for humans to see). However, analytics will only outperform the status quo in a subset of cases.
Access to statistical expertise was not the primary constraint in this coalition, and we suspect that technical expertise is not actually the primary reason that predictive analytics go unused. Only a few weeks of a data scientist’s time was required by each partner for the analyses described above. The IPI insight from MomConnect data was revealed by generating summary statistics that did not require specialized data science skills. Overall, technical expertise is important, but not in the quantity that one might assume.
Rather, the primary constraint has been attention - especially of leadership and managerial staff. In the hustle of implementing programs and raising funds, non-profit leaders find it hard to devote the patient, concerted effort required to make incremental progress. And outside academic researchers often find these small refinements too trivial to work on. We found it catalytic to combine the endorsement of leadership with a bit of consultative time, and modest financial investments. In other words, we just needed to connect a few dots, over time. We want to devote continued attention to this work, to achieve the prize of impact, and encourage others to be prepared to do the same.
Third, a combination of global and local expertise is very helpful. Contextual knowledge and subject-area expertise can be crucial to identify the most powerful insights and next steps. For MomConnect, despite investing weeks of structured brainstorming, it was not obvious which health-related outcomes should be prioritized. Input from an external global health expert - Dr. Jessica Cohen - prompted us to investigate IPI, which was an outcome we had not otherwise considered. Ultimately, our IPI-focused analyses have proven to be the most successful and important to date, and are sufficient to prompt important downstream program actions.
D-Tree’s algorithm identified pregnant clients who were unlikely to deliver in health facilities or with skilled attendance. However, it was not obvious how to support these clients, especially given variations in health facility quality and capabilities across Zanzibar. Local expertise was critical for learning that a substantial initiative to measure and reinforce facility quality was already planned. Local knowledge was also crucial to rule out some evidence-based interventions (e.g. incentives to deliver in health facilities) and instead identify a highly promising, locally developed, and more cost-efficient intervention to pursue: community-based group antenatal care.
In short, we have learned that data cannot reveal "great truths" without additional input and interpretation. Both outside expertise and local knowledge have been crucial complements to unlock greater impact.
Finally, partnerships thrive with on-call support and peer encouragement. “Collaborative partnership” may sound fuzzy and wishy-washy, but open collaboration across the partners in our coalition proved highly valuable, for several reasons.
- Peer pressure and deadlines quickened our progress as a group. Routine data analyses are rarely the most urgent and are easily deprioritized. It seems trivial to send a group message announcing that “everyone else will be ready with their regressions this week - will you?”. Yet it is incredibly effective.
- Ideas and capabilities were freely shared across our coalition. D-Tree and Living Goods share a similar program model and an identical technology platform, making it straightforward to share analyses and insights. But they also brought an abundance mindset, recognizing that a “win” by one organization is a win for others as well. Similarly, the participation of global public health experts in our coalition has benefitted all partners, including the experts themselves. Complementarities are unlocked through a partnership approach.
- Mobilizing pooled resources has, so far, proven efficient for our fundraising efforts, relative to fundraising as independent organizations. We had a centralized proposal writing effort, rather than five independent efforts. And it is easier to create a sum that is greater than its parts when operating as a partnership by default.
Moving forward, we plan to experiment with other ways to facilitate partnership and collaboration in the perinatal agency space. This includes writing more learning pieces like this, and running collaborative sprints and gatherings to make efficient work progress. We also hope to launch a fellowship to support additional organizations interested in leveraging predictive analytics to expand women’s agency. Overall, we feel that data-driven program refinements can be widely adopted by non-profit organizations, if we make them social, collaborative, accessible, and routine.
___________
1. MomConnect RSA supports 57,000 registered adolescents. The WHO recommends 24 months between pregnancies.
2. Observational studies reveal that short IPI’s are associated with pre-term birth and low birth weight. We could not find research investigating impacts on siblings. Overall, IPI appears to be an important area that would merit further research. See select citations here, here and here.
3. D-Tree’s program currently enrolls ~30,000 pregnant women per year, with this number expected to increase as the program matures. Ten precent of enrolled pregnant women are identified with a 20-40% likelihood to deliver in a health facility. Facility delivery is associated with reduced mortality and morbidity for both mother and child. For example: "Health facility delivery is found to reduce the risk of neonatal mortality by 29% in low and middle income countries".