Evaluation of the Rheumatic Fever Prevention Programme

[Published online: 03 April 2013
Full report: implementationrheumaticfeverpreventionprogrammeapr13.pdf]

Implementation and Formative Evaluation of the Rheumatic Fever Prevention Programme

FINAL Report

Prepared for Ministry of Health Manatū Hauora

15 February 2013

[…]

1 Executive summary

1.1 Background to the Rheumatic Fever Prevention Programme

Rheumatic fever is a preventable disease that occurs in some people following an infection of the throat with Group A Streptococcus (GAS). If left untreated the infection can lead to an autoimmune response that may permanently damage the heart through the development of rheumatic heart disease (RHD). Rheumatic fever and RHD is a disease that disproportionately affects Māori and Pacific children, and is a marker of both socioeconomic deprivation and poor access to healthcare.

Recognising the social and financial cost of rheumatic fever to the New Zealand population, the Government committed resourcing from both Budget 2011 ($12 million) and Budget
2012 (an additional $12 million) for the Rheumatic Fever Prevention Programme (RFPP). The overarching goals of the RFPP are to achieve equity of incidence and outcomes of rheumatic fever between Māori and Pacific children, and other children, and reduce the
overall rates of the disease.

The objectives of the RFPP are to establish school-based throat swabbing services in targeted schools in high-risk areas; develop clinical tools and training to support the health sector; develop a health promotion and awareness programme (with high-risk communities, the health sector and the general public); and develop a rheumatic fever surveillance and monitoring programme.

RFPP funding was allocated to throat swabbing providers within eight District Health Board
(DHBs) areas based on cluster analysis of rheumatic fever incidence, specifically:
Northland, Counties Manukau, Waikato, Bay of Plenty, Lakes, Tairāwhiti, Hawke’s Bay, and
Capital & Coast. As specified in their contract, the overall service objective for these providers is to reduce the rate of rheumatic fever by:

Providing throat swabbing and referral services in the school, home or other settings as appropriate for school children aged 5–14 years who present with sore throats in high- risk areas, and eligible whānau/family members living with these children.

Increasing awareness of rheumatic fever risk factors among children and their
whānau/families in the key geographical areas.

Developing and maintaining relationships with other health and social service providers
(including Whānau Ora providers) to facilitate referral and support, as appropriate.

In 2011 (after the launch of the RFPP), new rheumatic fever targets were announced as part of the Better Public Services (BPS) Action Plan. The RFPP was integrated into this broader inter-agency government work programme from September 2012, acknowledging the need for a multi-faceted approach to reduce the high rates of rheumatic fever in New Zealand.

1.2 Evaluation overview

Litmus was commissioned by the Ministry of Health (‘the Ministry’) to undertake an implementation and formative evaluation of the first 18 months of the RFPP from 1 July
2011 to 31 December 2012. The key aims of the evaluation were to provide an overview of how the RFPP is being been implemented, and recommendations to enhance
implementation.

The evaluation used a mixed-methods approach including: literature and documentation review; key informant and RFPP provider interviews; review of monitoring data; case studies of four RFPP sites (Whāngarei, South Auckland, Rotorua and Porirua); 12 focus groups with parents/caregivers; and a school intercept survey of 407 parents/caregivers. These data sources were analysed collectively to address the evaluation aims and objectives.

As this is a formative and implementation evaluation, it is acknowledged that the plans and activities of the Ministry, RFPP providers and other agencies have evolved and progressed since the fieldwork period and the completion of this report.

1.3 Evaluation findings

Sore throats are not a priority for parents/caregivers

Parents/caregivers interviewed are committed to fostering the health and well-being of their family/whānau. Parents/caregivers seek to be proactive in supporting their children, and in preventing illness and disease. However, they face a number of inter-related environmental, social and economic barriers that undermine their ability to support the wellness of their family/whānau, in particular cold and damp homes.

For parents/caregivers interviewed, sore throats are not a priority compared to other health
needs of children and family/whānau. Their usual response to a child’s sore throat is to
‘keep an eye on’ their child, and usually they will attend school. Medical care was only sought when the condition got worse or coincided with another more urgent concern.

Costs, available appointments and long wait times deter parents/caregivers from taking their child to a doctor with what is perceived as a relatively minor disorder. Awareness of the link between untreated GAS throat infection and rheumatic fever is low.

Positive and relevant initiative

The RFPP throat swabbing service fills a need for those most at risk of rheumatic fever, addressing many of the barriers to diagnosis, treatment and preventing progression from an untreated sore throat to rheumatic fever.

Parents/caregivers are overwhelmingly positive about throat swabbing being delivered in schools as the service does not cost them anything, is convenient, undertaken by trusted providers in a place where children are comfortable, and the results and provision of appropriate treatment is done in a responsive way. The throat swabbing service has been well received by schools, with principals, teachers and administrative staff supportive of RFPP provider visits and the service they are providing for students.

Local providers and wider stakeholders are acutely aware of the damage rheumatic fever creates in their communities, and the RFPP is a timely initiative. While DHBs are supportive of initiatives to address rheumatic fever in their regions, they advocate for the RFPP to be linked and integrated with their existing initiatives and wider services. This integration of services is perceived to improve the chances that the RFPP goals will be achieved, and be more sustainable long-term.

Throat swabbing is occurring amongst high-risk populations

As contracted, all RFPP sites have implemented their throat swabbing services. A range of service delivery models for throat swabbing are being used that reflect regional and community needs, and the capacity and capability of the RFPP provider. Providers are at different stages of implementation, reflecting the staggered commencement dates.

RFPP providers, across the four case sites, have a highly dedicated frontline workforce, with strong community links with families/whānau, schools and local provider networks, including sub-contracted providers delivering other aspects of the RFPP such as home visits for antibiotics delivery, compliance checks and health promotion. Providers are adopting innovative and culturally appropriate practice to ensure engagement with high-risk populations.

During the early implementation stages, local providers have primarily focused on establishing the school throat swabbing component of the RFPP. Community awareness raising and primary care engagement aspects of the RFPP have received less attention.

Strengthening RFPP delivery

A number of areas were identified that would strengthen the delivery of the RFPP, specifically:

Delivering across all RFPP workstreams in particular community awareness raising and engagement with primary care. Focus in these areas is critical in ensuring sustainable outcomes for parents/caregivers after the completion of the RFPP. Parents/caregivers need to be informed on when to act and to be empowered to access appropriate services.

- Community awareness raising - Feedback from parents/caregivers highlights the need for communications that are engaging, culturally appropriate, and supplemented by face-to-face opportunities to ask questions, particularly for Māori and Pacific parents/caregivers. For many parents/caregivers communications alone will not support behaviour change due to health literacy and a range of other
barriers. For these parents, health systems need to actively facilitate their access to diagnosis and treatment as part of the RFPP.

- Engagement with primary care – RFPP provider engagement and awareness raising with primary health care providers is minimal across the sites, and usually relies on individual initiative rather than a systematic process. Health professionals in RFPP areas have a key role in reinforcing core messages and behaviours in their interactions with the community.

Ensuring effectiveness for tamariki Māori and their whānau – While the RFPP has a strong focus for tamariki Māori and their whānau, services could be strengthened. Suggestions include ensuring providers across all regions have the capability to reach those most at-risk from rheumatic fever, and have local credibility for frontline services and home visits. Providers also need CAR resources that have strong relevance for Māori, and do not stigmatise tamariki and their whānau.

Ensuring effectiveness for Pacific children and their fanau – Currently, there is limited focus on Pacific people at strategic and service delivery levels for the RFPP, despite a high Pacific population in two RFPP sites. Given the considerable burden of the disease on Pacific children and the objectives of the RFPP, this lack of focus is a concern.

Developing referral pathways with other health and social support services to address the underlying causes of rheumatic fever – There is wide variation across RFPP providers in facilitating referral and support of children and their families/whānau with other health and social service providers (including Whānau Ora), where needed. Throat swabbing services are being delivered (in some areas) in a way that does not identify nor follow up child health or social issues. The lack of clear process presents a significant risk for RFPP consistency, and may miss crucial opportunities to address
the underlying determinants of rheumatic fever, and improve child and family/whānau
health more broadly.

Need to enhance the RFPP’s focus on quality improvement. Wide variation in approaches to service delivery exists across providers at the local level. A degree of flexibility is important for providers to respond appropriately to meet unique local needs. However, this flexibility needs to be embedded in clear operational frameworks and guidelines for the RFPP service delivery with regard to: consent approaches; the use of standing orders; budgeting and payment for pharmacy, GP and laboratory costs; swab quality and swabbing processes; protocols for community-based/opportunistic
swabbing and referral pathways.

Efficacy of RFPP in reducing rheumatic fever

Experts and providers expressed concern about the efficacy, cost-effectiveness and sustainability of reducing rheumatic fever using the current mix of RFPP activities, particularly as primordial determinants of health and preventing GAS infection are not addressed by current services. Widening the scope of deliverables and strengthening referral pathways to wider health and social services will support providers to build on progress made in the early stages of the RFPP, and potentially address underlying determinants of rheumatic fever.

Measuring outcomes and impacts

A more systematic approach to impact and outcomes measurement for the RFPP is needed, with assurances of quality data collection. The Ministry is currently completing the monitoring and evaluation framework and streamlining the provider monitoring requirement to ensure quality data. Stakeholders advise that identifying target areas (and allocating funding) based on GAS incidence would be a more reliable indicator of community need. However, currently there are no national measures for GAS, and significant gaps in understanding of the epidemiology of GAS.

Planning for sustainable outcomes

Providers and stakeholders are advocating for collaborative and future-focused planning ensure RFPP outcomes are sustainable. Providers need adequate lead-in time for contract completion to manage the expectations of schools involved, and ensure parents/caregivers can access diagnosis and treatment for their children when the throat swabbing service withdraws. Awareness rasing for parents/caregivers , effective systems for families/whānau who are under-served by current health services, and engaging with primary care to
support rheumatic fever prevention will be key to sustaining RFPP gains long-term.

1.4 Recommendations

The following table details recommendations to enhance the RFPP as drawn from the evaluation.

Table 1: Implementation and Formative Evaluation recommendations for the RFPP Area Recommendations
1. RFPP focus Consider collection of GAS incidence data to more reliably inform location of future swabbing services
Consider a programme that targets GAS prevention in the community, rather than individual symptomatic throat swabbing in children
Continue and widen focus on addressing primordial determinants of health in target areas
2. Pacific Strengthen focus on Pacific from strategic (Ministry) level through to contracts and service specifications (especially for service delivery and CAR in areas with Pacific populations)
Ensure Pacific-language resources are made available in a timely manner
Consider production of other appropriate CAR resources and materials, including non-written media
3. Māori Seek clarity on requirements of sub-contracted providers around service
appropriateness for Māori
Consider production of appropriate CAR resources and materials, including non-written media

4. Service integration

5. Primary care engagement

6. National and local CAR

7. Quality frameworks and guidelines

Examine options for integrating throat swabbing with other child health services
Ensure referral pathway model/protocols developed (with health and wider social support agencies)
Involve DHBs in future funding decisions to ensure RFPP planning reflects local knowledge and local strategies
Focus effort on building primary care awareness and engagement, with
RFPP and wider rheumatic fever messaging
Establish a centrally-led primary care awareness raising strategy
Review focus and allocation of spend to HPA in out-years
Establish strategic national RFPP CAR plan
Review respective national (HPA) and local (provider) community awareness raising roles and responsibilities and fund accordingly
Ensure national-level CAR learnings from other programmes targeting
Māori and Pacific are considered in the development of future resources
Ensure timely production of any future resources
Ensure resources are appropriate for the local context
Provide national guidance and support on:
- use of standing orders
- the use of an ‘opt out’ consents approach
- referral pathway model/protocols (including health and wider social support)
- community-based/opportunistic swabbing guidelines and protocols
- use of sub-contracted providers (ie, ensuring appropriateness of
service delivery for Māori and Pacific children and families/whānau)
Note: some of the above could be delivered through NCS.

Area Recommendations

8. National Coordination Service

Strengthen NCS’s role to:
- establish quality assurance standards and monitoring for swab taking and ensure consistency in quality within and across sites over time
- advise providers on how to implement swabbing approaches and processes (informed by standards developed for school immunisation programmes, for example)
- advise on and recommend use of standing orders
- create opportunities to share learnings between frontline staff
- regularly document and disseminate best practice approaches

9. Sustainability Ensure a transparent exit plan is communicated to providers ahead of contract end date (at least six months)
Clarify expectations of relative emphasis on CAR versus throat swabbing with local providers
Consider links to other child health programmes with the aim of integrating service delivery and messaging opportunities

10. Future evaluation

Review provider data monitoring for content and quality to ensure longer- term RFPP outcomes will be measurable over time
Establish a monitoring and evaluation framework to reliably inform effectiveness of RFPP delivery and tracking of RFPP progress

[…]

[Published online: 03 April 2013
Full report: implementationrheumaticfeverpreventionprogrammeapr13.pdf]

© Scoop Media