Why a process evaluation?July 25, 2014
A process evaluation, however, is a vital part of scientific studies such as the randomized controlled trial or RCT being conducted through the BC Healthy Connections Project (BCHCP).
A kind of a study of a study, a process evaluation determines how well an intervention is being implemented and what factors may be influencing the outcomes. In the case of the BCHCP, for example, the NFP intervention — home visiting by public health nurses or PHNs — is being delivered to young, low-income women who are preparing to parent for the first time.
But NFP is being delivered in very diverse communities across BC, notes Susan Jack, who is principal investigator for the study’s process evaluation, which is being funded by the Public Health Agency of Canada. “What happens when the program is delivered in smaller communities where there may be more barriers to travel and communication? And are there differences in delivering NFP between urban and rural areas in BC?” she asks. “We’re going to try to understand all these variables.”
NFP, which starts early in pregnancy and continues until children reach their second birthday, has proven successful over 35 years in the US. But this current RCT is the first scientific evaluation of the program in Canada. (NFP’s founder, David Olds, explains why an RCT is necessary.)
The primary participants in the process evaluation, however, won’t be the women and children. They’ll be the PHNs and nursing supervisors who are responsible for NFP visits. “We’ll be interviewing them every six months to explore where they are with the program, the challenges that have come up, their perceptions of the education they’ve received and any gaps that may exist,” Jack says.
The process evaluation will also study how the program is being implemented in five different Health Authorities to meet the needs of families across the province. One of its most important functions is to determine whether the program is being delivered with fidelity to the NFP model,” Jack says, mentioning the high frequency of visits that mothers are offered as an example. “If that’s not happening then we need to know why.” More broadly, the process evaluation will also help us understand BC’s unique context – including the situations faced by PHNs, their supervisors and the families they visit – and the solutions developed by the NFP teams to resolve any emergent challenges.
Another issue for the process evaluation is refining the NFP model to meet the unique needs of young, disadvantaged first-time mothers in BC. These young women are often hard to reach because they may not have stable housing, reliable work or supportive extended families – at a time in their lives when they need a lot of support. PHNs in turn may find it challenging to locate young people who may be highly mobile. Financial difficulties may also lead to some women not having easy access to phones. For example, “many families use phone cards and once the phone card runs out, they don’t have land lines,” Jack says. “Many of the young moms we work with go from family to family, and many of the nurses will struggle with trying to locate them.”
These young women may also not have had good role models to help them learn what a trusting relationship looks like or how to maintain healthy relationships. This will potentially make building a trusting relationship with their NFP PHN more challenging. This is yet another issue that the process evaluation will investigate.
Jack has high hopes for the process evaluation and high praise for the BC PHNs she has met. “The NFP nurses in BC are among the most skilled I’ve ever worked with,” she says. “We’re still refining everything and testing the tools and the procedures. But our work with the BC process evaluation will be a huge benefit for the international NFP community.”
Note that NFP is available only through the BC Healthy Connections Project (BCHCP) for the duration of recruitment. Practitioners or young pregnant women can click here for details on how to reach public health and determine eligibility for the BCHCP.