Post-Ebola Community Health Worker programme performance in Kenema District, Sierra Leone: A long way to go!

Introduction

The effect of the 2014–2015 West African Ebola outbreak on Community Health Worker (CHW) services in Kenema district, Sierra Leone, was assessed through a retrospective cross-sectional study comparing CHW reporting and services before (June 2013-April 2014), during (June 2014-April 2015) and post-outbreak (November 2015-April 2016)¹. The study found CHW reporting completeness and reported treatments for malaria increased post-Ebola, while those for pneumonia and diarrhoea returned to pre-outbreak levels. Results showed CHWs stopped performing Rapid Diagnostic Tests (RDTs) during the Ebola period and did not resume until after the outbreak. The study recommended additional investigations into preliminary positive trends in the early post-Ebola period to determine if these post-outbreak tendencies represent a resumption of pre-Ebola CHW programme performance or a continued progression. Therefore, we aimed to understand whether health system performance was sustained 18 months after the outbreak by reporting the trend and comparing CHW system performance (reporting completeness, reported diagnosis and treatment services among reports received) in Kenema district in the pre-, intra- and post-Ebola recovery periods for children under five years.

Methods

This was a retrospective cross-sectional study using aggregate CHW programme data from Kenema district, Sierra Leone. Details of Kenema and the CHW programme were previously described¹. The study population included all CHW programme reports from Kenema Peripheral Health Units (PHUs) during the pre- (June 2013–April 2014), intra- (June 2014–April 2015), and post-Ebola recovery (June 2016–April 2017) periods. Data during the month of May 2014 were excluded to prevent potential spill-over effects across periods. Consistent months were deliberately chosen to allow for seasonal comparison across the three periods.

CHW programme data were extracted from the electronic Ministry of Health and Sanitation (MoHS) District Health Information Systems (DHIS2), which is aggregated from monthly summary sheets submitted by PHU supervisors. Data on the following variables were extracted per month: facility reporting completeness; reported malaria RDTs; treatment for malaria, diarrhoea and pneumonia. The data available did not include variables on number of CHWs per facility. In this study we defined CHW reporting completeness as the proportion of facilities with fully (100%) complete submissions of CHW monthly summary sheets expected for the time period. We imported data into STATA v14.2². We produced descriptive summary statistics and examined statistical differences between periods using two-sample t-test, with significance set at 0.05. We undertook a trend analysis for the periods pre- (June 2013-April 2014), intra- (June 2014–April 2015), and post-Ebola recovery (June 2016–April 2017) periods.

The national distribution of RDT and treatment supplies, which are distributed at the district level through the District Health Management Team’s (DHMT) district store, are supported by the Global Fund. Each DHMT is expected to allocate 30% of the RDTs and drugs to CHWs in the district. Financial incentives for CHWs differ across different districts, depending on the partner supporting the programme in that specific district. In Kenema district, the Global Fund supports the programme and CHWs are provided with a financial incentive of SLL 100,000 per month, as per the National CHW Policy³.

While we do not have data on the number of CHWs in Kenema district, based on population data⁴ and estimates on CHWs⁵, we estimate that the district has roughly 1,250 CHWs.

Supervision across the CHW programme operates at different levels. CHW peer supervisors are identified out of every 10 CHWs, with the aim to offer supervision at the PHU level. At the district level, CHWs meet monthly with the PHU in-charges at the DHMT. Supportive supervision is provided from national level on a quarterly basis and which is supported by an assessment tool. During the Ebola outbreak period, the CHWs operated under the supervision of the District Social Mobilization Coordinators and were detecting suspected Ebola cases using the community case definitions⁶ and childhood illnesses using revised guidelines for the Integrated Community Case Management of Childhood Illnesses (ICCM)⁷.

The Sierra Leone Ethics and Scientific Review Committee (dated 18 December 2018) and the Ethics Advisory Group of the International Union against Tuberculosis Lung Disease (UAG number 65/18) provided ethics and institutional approval. Since aggregate programme data were used, participant consent was not sought.

Results

Proportion of PHUs reporting complete reports

Figure 1 shows the proportion of facilities with CHW complete reports monthly in the pre-, intra- and post-Ebola recovery periods. The number of PHUs submitting complete CHW reports ranged between 39–129 (27–89%) and 98–134 (68–92%) in the pre- and intra-outbreak periods respectively. However, there were some missing reports. Conversely, in the recovery period, 136 (94%) facilities consistently completed reports each month.

Figure 1. Proportion of Health Facilities with complete Community Health Workers reports in the District Health Information System, Kenema District, Sierra Leone, in the Pre-, Intra-, and Post-Ebola Recovery Periods.

CHW – community health worker. Pre-Ebola Period – June 2013 – April 2014, Intra-Ebola Period – June 2014 – April 2015, Post-Ebola Recovery Period – June 2016 – April 2017.

Mean monthly RDTs, malaria, diarrhoea and pneumonia cases reported

Figure 2 shows trends across pre-, intra- and post-Ebola recovery periods for CHW programme performance indicators. Table 1 shows that the mean monthly reported RDTs had an increasing trend in the pre-, intra- and post-Ebola recovery periods, with increases of 11% intra-Ebola and 35% (p-value=0.012) post-Ebola compared to the pre-Ebola period. The mean monthly malaria treatments reported trends similar to RDTs with increases of 31% (p-value=0.026) and 66% (p-value=0.020) intra-Ebola and post-Ebola, respectively.

Figure 2. Community Health Worker programme performance indicators for children aged under 5 years in Kenema district, Sierra Leone, in the Pre-, Intra-, and Post-Ebola Recovery Periods.

RDT - Rapid Diagnostic Test. CHW – community health worker. Pre-Ebola Period – June 2013 – April 2014. Intra-Ebola Period – June 2014 – April 2015. Post-Ebola Recovery Period – June 2016 – April 2017.

Table 1. Reported monthly mean values and differences across periods in reference to the Ebola outbreak for performance of Community Health Worker Programme in Kenema District, Sierra Leone.

Reported monthly mean values	Period			Difference between pre- and intra- Ebola periods				Difference between pre- and post- Ebola recovery periods
	Pre- Ebola	Intra- Ebola	Post- Ebola
	n	n	n	n	%	95% CI	p-value	n	%	95% CI	p-value
Reporting completeness	79	85	94	6	8	(6, 19)	0.29	15	19	(4, 27)	0.012
RDTs	5877	6538	7963	661	11	(-1174, 2494)	0.46	2086	35	(369, 3802)	0.020
Malaria treatment	5338	6994	8882	1656	31	(222, 3092)	0.026	3544	66	(2029, 5060)	<0.001
Diarrhoea treatment	1083	627	866	-456	-42	(-106, -804)	0.013	-217	-20	(-528, 95)	0.16
Pneumonia treatment	699	625	1258	-74	-11	(-370, 223)	0.61	559	80	(197, 921)	0.004

CI: confidence interval; RDT: rapid diagnostic test. Pre-Ebola: June 2013 – April 2014; Intra-Ebola: June 2014 – April 2015; Post-Ebola Recovery: June 2016 – April 2017.

The mean monthly diarrhoea treatments reported exhibited a downward trend, in the pre- and intra-Ebola periods, followed by a small uptick in the post-Ebola recovery period. However, compared to the pre-Ebola period, overall monthly mean reported treatments significantly decreased by 42% (p-value=0.013) in the intra-Ebola, while the reduction in the post-Ebola recovery period was not statistically significant (p-value=0.16).

The mean reported monthly pneumonia treatments declined by 11% intra-Ebola but grossly increased by 80% (p-value=0.004) during the recovery period compared to the pre-Ebola period (Table 1).

Discussion

Our study results indicate a general trend of the CHW programme in Kenema district sustaining comparable performance levels into the post-Ebola recovery period as compared to pre-Ebola for reporting completeness and reported RDTs, malaria and pneumonia treatments. Comparable to results from the previous study¹ and the region⁸, this may be due to sustained investment in key areas affecting supply and services. The Integrated Disease Surveillance and Response approach, which emphasises active community-based surveillance and completeness of reporting, was strengthened in the intra-Ebola period⁸. In addition, supportive supervision, a critical element to the success of CHW programmes⁹, was revived after the outbreak⁸. Finally, financial incentives for CHWs initiated during the outbreak were subsequently incorporated into the national policy in 2016³. Furthermore, the CHWs have reported increased community awareness of signs and symptoms of common childhood illnesses, translating into increased demand for services⁸.

We found fewer missing reports in the pre- and intra-Ebola periods compared to the previous study¹, most likely due to retrospective data entry into the DHIS2. While we found higher mean reported RDTs performed in the post-Ebola period, we still observed fewer in the intra-Ebola period corresponding with the enactment of the “no touch policy,” similar to the previous report^1,7,8.

Our results reveal an absolute decline in the reported diarrhoea treatments during the recovery period. The promotion of hygiene practices through community sensitization^9,10 and institution of bye-laws by community stakeholders in the intra-Ebola period may have had lasting effects on behaviour and thus contributing to the reduction in reported diarrhoea treatments during the post-Ebola period⁸. However, the difference in reported treatments between pre- and post-Ebola periods was not found to be significant, so this trend may warrant future investigation.

Utilising pre-Ebola service levels for comparisons, while useful, still reflect benchmarks of weak systems functioning, a factor which contributed to the impact of the Ebola outbreak itself¹¹. Therefore, it is imperative that such comparisons be evaluated in this light. Therefore, in order to achieve the vision of the Sustainable Development Goal (SDG) for good health and wellbeing beyond pre-Ebola-level benchmarks, sustained investments in supportive supervision and financial incentives for CHWs are essential for the programme.

A strength of the study is the use of complete district PHU data for the study period. Furthermore, we adhered to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines for the reporting of observational data and sound ethical principles¹². Primary weaknesses of our study were the use of routinely collected data, which was influenced by reporting completeness. In addition, we defined reporting completeness per facility rather than per CHW due to availability of data. The use of routine data also led to a lack of data triangulation; inability to validate the electronic data base against the raw data and to generalise to the national CHW programme. In addition, we were not able to include data on the number of CHWs per facility, as they were not available.

In conclusion, although our study established a sustained trend towards the pre-Ebola CHW service levels 18 months after the outbreak, there is need for continued investment in the CHW programme to continue gains in programme performance in order to contribute towards SDG 3.

Data availability