The protective effectiveness of control interventions for malaria prevention: a systematic review of the literature

Introduction

During the 2000s, several malaria control interventions have been largely adopted and scaled up in endemic countries. These interventions mainly include long lasting insecticidal nets (LLIN), indoor residual spraying (IRS), use of rapid diagnostic tests (RDT) to improve malaria diagnosis and artemisinin-based combination therapy (ACT) as first-line for the treatment of uncomplicated malaria, intermittent preventive treatment of pregnant women (IPTp), intermittent preventive treatment for infants (IPTi), and seasonal malaria chemoprevention (SMC). This considerable deployment of malaria control interventions has largely benefitted from the increase in international funding for the fight against malaria¹.

Before being scaled up, the efficacy of control interventions needs to be demonstrated in controlled trials (phase III). These trials consist generally in randomizing individuals or clusters, half receiving the intervention tested, and the other half receiving a control, i.e. a placebo or the best intervention available. Both the intervention and the control are strictly delivered under monitored conditions, in order to preclude biases in the estimate of the efficacy. For any given intervention, especially life-saving interventions, a small number of trials are conducted, because once the efficacy or the superiority of the intervention has been demonstrated it would be unethical to keep providing a less effective intervention to the population, and also because of the important resources needed to conduct such trials. These studies are indispensable to ensure that people receive best interventions available but they poorly depict what will be the effectiveness of the intervention once deployed in real life, for two reasons: (i) phase III trials are conducted in a limited number of settings that don’t encompass all possible field conditions, and (ii) special attention is paid in delivering the intervention during controlled trials, thus rendering ‘ideal’ conditions.

National malaria control programs and donors should nevertheless make sure that the effectiveness of the interventions is confirmed once they have been deployed on the field. As a matter of fact, detecting suboptimal effectiveness of control interventions is critical for policy guidance. This is becoming particularly important since the global budget of the fight against malaria ceased growing¹ and thus funding tends to be allocated for most effective interventions. The word “effectiveness” actually encompasses three concepts^2,3: the coverage (does the intervention reach the population?), the individual protective effectiveness (does the intervention protect against or treat the disease?), and the community protective effectiveness (does the intervention benefit for others than the very ones who received the intervention?). Surveillance data and ecological studies can measure the overall impact of the control policy but they won’t be helpful in determining whether each intervention taken separately yielded the expected impact since all interventions are usually implemented concomitantly, and because they are influenced by environmental and social factors. Cross-sectional indicator surveys and social sciences studies will provide useful data regarding coverage of interventions and their determinants, but some data may be missing regarding the protective effectiveness (PE) of the interventions.

As mentioned above, it would be unethical - and laborious - to conduct controlled trials in all existing conditions and areas in order to verify the PE of malaria control interventions. Thus, alternative study designs must be applied. The PE can be evaluated (i) by biological studies measuring the bio-efficacy of drugs or insecticides (indirect measurement of the PE) or (ii) by epidemiological surveys yielding a direct measure of the PE of the intervention. The second approach encompasses three major study designs for phase IV assessment of the effectiveness of malaria control interventions, using historical non-compliant controls²: case-control surveys (CCS), cross-sectional surveys (CSS), and cohorts. The stepped-wedge design also allows for the evaluation of the PE, although it sometimes relates to the efficacy (phase III) when the implementation of the intervention is strictly overseen, and sometimes to the effectiveness (phase IV) when the intervention is implemented under field conditions. All these study designs share as a common drawback the possibility of biases: non-compliant controls are not strictly comparable to people having received and using the intervention. Adjusting for socio-demographic variables can reduce but not eliminate biases at the individual level, and adjustment variables can be challenging to identify and to quantify when it comes to the evaluation of the community PE (ecological bias).

In order to assess the disparities between efficacy and effectiveness of malaria control interventions, and the range of PE observed on the field, we conducted a systematic review of the literature on epidemiological studies providing data about the protective effectiveness of control interventions for malaria prevention (CIMP) under field conditions. This study also has as secondary objectives (i) to appreciate what were the study designs most used, (ii) which populations were most surveyed, and (iii) what was the representativeness of these studies.

Methods

The PubMed database was searched for references using an algorithm (provided in Supplementary File 1) looking for (i) keywords related to malaria and (ii) keywords related to control interventions for prevention and (iii) keywords related to study designs that allow for the measure of the effectiveness. Bibliographies of the articles identified were also examined to find additional reports. The search was run for the last time on June 23, 2015.

We intentionally excluded efficacy studies such as controlled trials in the context of phase III assessment, and studies that aimed at measuring indicators such as coverage or factors associated with the uptake of interventions. Studies aiming at measuring the bio-efficacy of interventions using other methods than epidemiological, were also excluded from the database. The present study focuses on intervention for malaria prevention: LLINs, IRS, IPTp, SMC, IPTi, larval source management, and information, education and communication (IEC) campaigns. Given that the use of other insecticides than IRS, such as repellents or mosquito coils, have recently demonstrated an interest in preventing malaria^4–6, their PE were also recorded. Articles presenting the effectiveness of IEC regarding prevention behaviours were included. Only articles in English, French, Dutch, or Spanish were considered.

We focused on two outcomes: (i) the measure of the PE against peripheral parasitemia, as measured by RDT and/or blood smears and/or PCR, and (ii) the measure of the PE against occurrence of acute clinical malaria. In studies having investigated other biological or clinical outcomes simultaneously, we recorded preferentially the two outcomes mentioned above. Whenever several PE results were available from a single study (e.g. for different subpopulations), all PE results were retrieved.

On the basis of the objectives of the study disclosed in the title, the abstract and the article, we determined whether the study was aimed at measuring the effectiveness of a CIMP or if this measure was done “accidentally”, e.g. for the purpose of controlling for other associations. The PE was defined as one minus odds ratio (OR) or one minus relative risk (RR), depending on the study design.

Results

Of the 1423 references retrieved, 523 were discarded on the basis of the title; 893 abstracts were checked and 683 of these didn't address the effectiveness of CIMP; seven abstracts could not be accessed (see flow diagram provided in Supplementary File 2). We thus identified 203 papers related to studies that aimed at measuring the effectiveness of CIMP or in which the effectiveness of CIMP was measured but 10 of them could not be accessed. One study was excluded because it focused on travellers and not resident populations of endemic areas; one reference was excluded because of the language; one study was excluded because it evaluated an intervention that was not particularly targeting malaria or mosquito control (cotrimoxazole in HIV positive pregnant women); two studies were excluded because it evaluated a CIMP out-dated (chloroquine chemoprophylaxis in pregnancy); 14 studies were excluded because no OR or RR value was presented and the data disclosed in the article didn’t allow for calculation of the PE of CIMP. Among the remaining 175 references, 13 presented methodological problems incompatible with the inclusion in the present review, such as absence of definition of cases or definition of the exposure to CIMP.

The final review included thus 162 studies. This included 133 (82.1%) studies on bed nets, 37 (22.8%) studies on IPTp, 25 (15.4%) studies on IRS, and 22 (13.6%) studies on other interventions (Figure 1). One third of the studies (52/162, 32.1%) addressed more than one CIMP. Regarding studies' design, 106 (65.4%) were CSS, 29 (17.9%) were CCS, 24 (14.8%) were cohorts, and 3 (1.9%) were stepped wedge (Figure 1).

Figure 1. Proportions of CIMPs investigated in the 162 studies included in the review, and the proportions of study designs used.

The number of publications related to the PE of CIMP increased considerably during the years 2000’s and stagnated since 2010 (Figure 2). Two fifths (41.4%) of these studies were not directly aimed at measuring the PE of CIMP.

Figure 2. Number of references by year of publication, and the proportion of studies whose aim was the measure of the effectiveness of CIMP.

The last year is truncated since the search was performed in June 2015.

Regarding the representativeness of the studies, two thirds (109/162, 67.3%) were conducted at the district level or below. Only one third of the studies that didn’t investigate the effectiveness of IPTp (43/125, 34.4%) included the whole population while the majority (71/125, 56.8%) included only children. Only 15 of these studies (12.0%) were conducted in the whole population and at a regional level (≥2 districts, province/region, or island) or above (national or multi-country).

In 18.1% (50/276) of the evaluations of PE retrieved or recalculated from the 162 studies included in the review, the association between the malaria outcome and the exposure to the CIMP was not adjusted on other variables (univariate logistic regression, two-by-two tables, etc). Since the adjustment on age is particularly important, we calculated the proportion of studies conducted in a population where the age of the oldest participants was ≥10 years older than the youngest, but for which no adjustment on age was done in the measure of the PE. We found that 38.9% of the studies conducted in such a population with heterogeneous age groups didn’t adjust the calculation of the PE for age.

PE of bed nets

The search retrieved 169 measures of the PE of bed nets in 133 studies (Supplementary File 3). Most of the time (82.2%), the exposure to bed nets was measured at the individual level, but in 23 cases (13.6%) it was done at household level (ownership or proportion of users) and in seven cases (4.1%) at cluster level. The majority of PE measurements involved insecticide-treated nets (ITN) or LLINs (42.0 and 12.4%, respectively) but in an important proportion of cases the definition of bed nets didn’t include impregnation or not (42.6%). In some instances (2.9%), the measurement was specifically done for non-impregnated bed nets (NIBN). Most PE evaluations used the Plasmodium infection as outcome (56.8%), especially CSS that accounted for 62.7% of study designs (Figure 3), or clinical malaria (31.9%), especially CCS that accounted for 20.1% of study designs; some used an obstetrical outcome (7.1%), and a few ones used the mortality as outcome (4.1%). Cohorts represented 15.4% of study designs and there were only three stepped-wedge designs (1.8%). More than a half of PE results (58.0%) were obtained from paediatric populations and 27.8% considered the whole population; the other studies (14.2%) were conducted on women of childbearing age.

Figure 3. Proportions of study designs and outcomes investigated for the evaluation of PE of bed nets.

Most of the results (60.7%) demonstrated a significant PE from bed nets use. However, 38.2% of results were not significant (Figure 4, Figure 5 and Supplementary File 3). In 14.2% of cases, the PE value was negative, i.e. a trend towards a risk increased, and 1.2% of results showed a risk significantly increased. The median PE was 36.0% (interquartile range [IQR] 14.0–54.0%), and this differed only marginally according to CIMP definition: median PE was 39.8% (IQR 20.2–50.3%) for LLINs, 30.0% (IQR 16.5–52.0%) for ITNs, 51.0% (IQR 35.0–51.0%) for NIBNs, and 36.0 (IQR 13.8–54.6%) for bed nets without further precision of their impregnation.

Figure 4. Forest plot of evaluations of PE of bed nets against infection.

Results without CI are not displayed. Box size is proportional to the sample size.

Figure 5. Forest plot of evaluations of PE of bed nets against other outcomes than infection.

Results without CI are not displayed. Box size is proportional to the sample size.

PE of IRS

The search retrieved 32 measures of the PE of IRS in 25 studies (Supplementary File 4). CSS survey design was largely predominant (90.6%) and three PE evaluations from CCS (9.4%) were observed. A third of studies (34.4%) considered the whole population while 59.4% were obtained from paediatric populations and 6.3% from women of childbearing age. Most of the time (78.1%), the exposure to bed nets was measured at the household level, but in seven cases (21.9%) it was measured at cluster level. Only 21.9% of PE measurements of IRS considered recent spraying (≤6 months before the survey or delay since last IRS round in months), and the rest considered IRS ‘last round’, ‘last year’, or even ‘ever’. Most PE evaluations used the Plasmodium infection as outcome (87.5%) and the rest considered clinical malaria (12.5%).

Half of results demonstrated a significant PE of IRS (median 28.5%, IQR 8.8–47.3%), but 43.8% of results were not significant and 6.2% of results showed a risk significantly increased (Figure 6 and Supplementary File 4). The PE value was positive in more than three results out of four (78.1%). Median PEs were comparable when considering recent (20.0%, IQR -2.5–41.0%) or older spraying (32.0%, IQR 9.0–46.0%).

Figure 6. Forest plot of evaluations of PE of IRS.

Box size is proportional to the sample size. Recent spraying: ≤6 months before the survey or delay since last IRS round in months.

PE of IPTp

Our search retrieved 40 measures of the PE of IPTp using sulfadoxine-pyrimethamine (SP) in 37 studies (Supplementary File 5). Among these 40 results, 16 (40.0%) compared any regimen versus no SP dose, 13 (32.5%) compared the standard regimen versus no IPTp, and the remaining 11 (27.5%) compared the standard regimen versus substandard regimen. Most PE evaluations used an obstetrical or neonatal (e.g. low birth weight) outcome only (45.0%) or an outcome considering an obstetrical event or a maternal peripheral parasitemia (7.5%). The detection of Plasmodium in the mother’s blood was used as the outcome in 37.5%; three results (7.5%) had evaluated clinical malaria, and one used the mortality as outcome (2.5%). CSS represented 85.0% of study designs, cohorts 10.0% and there were only two CCS (5.0%). Most results (90.0%) were obtained from mothers, usually pregnant women at antenatal consultation and/or women at delivery units, but 10.0% considered paediatric populations (neonates or infants). The vast majority of studies were conducted at the district level or below (85.0%).

Most results demonstrated a significant PE of IPTp (median PE 49.0%, IQR 23.0–67.3%), but 32.5% of results were not significant and 5.0% of results showed a risk significantly increased (Figure 7 and Supplementary File 5). Median PE was 24.7% (IQR 4.0–70.0%) in studies evaluating standard IPTp regimen versus no IPTp, 50.5% (IQR 30.0–65.0%) in studies comparing any IPTp regimen versus no IPTp, and 50.0% (IQR 35.8–65.4%) in studies evaluating standard IPTp regimen versus substandard IPTp regimen. These values were comparable between studies evaluating the PE of IPTp against infection (median PE 43.0, IQR 7.0–78.5%) and studies evaluating the PE of IPTp against obstetrical outcomes (median PE 53.5, IQR 33.3–66.2%).

Figure 7. Forest plot of evaluations of PE of IPTp.

Box size is proportional to the sample size.

PE of the domestic use of insecticides

Our systematic search identified 20 evaluations, from 16 studies (Supplementary File 6), of the PE of the use of other insecticides than IRS, including coils (45.0%), sprays (30.0%), and repellents (10.0%). In the remaining 15.0% of cases, the PE of two or three of these formulations together was evaluated. The vast majority of these results came from CCS evaluating clinical outcomes (15/20, 80.0%), and the four other results came from CSS evaluating clinical (1/20) infection (1/20), or obstetrical (2/20) outcomes. The majority of these studies were conducted in paediatric populations (65.0%), some in the whole population (20.0%) and the remaining 15.0% among adult women.

Overall the PE of these insecticides was demonstrated in only four studies, whatever the formulation in coils, sprays, or repellents, and most (70.0%) results were non-significant (Figure 8 and Supplementary File 6). The median PE was 19.1% (IQR -21.0–38.5%).

Figure 8. Forest plot of evaluations of PE of domestic use of insecticides.

Box size is proportional to the sample size.

Discussion

This review showed that the efforts made for the evaluation of the effectiveness is increasing with time, in parallel with the global funding available for malaria control. Nevertheless, the number of published studies about the effectiveness of CIMPs seems to be stagnating since 2010. This could hinder the progress towards more cost-effective control policies, as the strategy should be locally adapted depending on data about the effectiveness of CIMP.

Overall, there is a sense of a low representativeness of the studies. Only one third of the studies were conducted at a large scale, and only one third included all ages and genders; only one out of eight had both features. Several CIMPs target the whole population of a region or a country, e.g. IRS or universal distributions of LLINs. When evaluating the PE of such CIMP, it’s crucial not to leave aside a part of the population since the effectiveness of CIMP may vary depending on transmission or between age groups for example^67,168,169. On the contrary, nearly 40% of studies conducted across age groups didn’t include the age in regression models while age influences both malaria outcomes (e.g. probability to be infected) and CIMP coverage¹⁷⁰. In order to yield unbiased evaluation of the PE of a CIMP, it is critical to adjust the measure of the association of malaria outcome and exposure to CIMP for age, as well as for other variables known to influence the outcome (e.g. socio-economic status, parity, rural or urban area) and to take into account the intra-cluster correlation in multi-stage sample designs. Moreover, several studies conducted at a large scale didn’t stratify the analysis. Since local features of malaria transmission or cultural behaviours may affect (or enhance) the PE of CIMPs, omitting stratified analysis precludes the identification of clusters where the effectiveness of CIMPs was suboptimal.

On the other hand, the multiplicity of local evaluations of the PE of CIMP offers an appreciation of the diversity of local conditions. Certain studies revealed that the PE was largely above what had been demonstrated in efficacy trials; this can result from biases inherent to observational studies, but it’s also possible that local conditions favour the effectiveness of CIMP, e.g. the PE of LLINs is expected to be especially high where vector populations exclusively bite indoor and late at night. Conversely, many studies failed to demonstrate the PE of CIMP studied or showed that it was lower than expected. This is where the interest of these surveys stands, for it urges policy makers and their research partners to investigate the causes of this failure and to propose alternative control interventions. This is also why we didn’t conduct a meta-analysis on the data retrieved in this review. Besides this, various meta-analyses of CIMP already exist, either reviewing efficacy studies only^169,168,171, or mixed both efficacy and effectiveness studies¹⁷².

This review presents several limitations, including the search in one database only, the limits in languages considered (although one reference only was discarded for this reason), and the incomplete access to articles. This review is thus probably not exhaustive but it intends to be largely representative of effectiveness studies. On purpose, we didn’t include meta-analyses of the studies included as the overall objective was to get a sense of what kind of studies had been done, and to be strictly descriptive on the results obtained. Our take-home message is not so much that MCI are effective on average, but that their effectiveness might be locally lower -or higher- than what is expected by efficacy studies.

Conclusions

This review shows that there is an increasing interest in measuring the PE of CIMPs. Most studies confirmed the PE of the CIMPs that they were evaluating, but an important part yielded a ‘negative’ PE and/or non-significant confidence interval. In this case, complementary investigations are needed in order to confirm the existence of a problem in the effectiveness of the CIMP and to propose alternative control measures if necessary.

A frequent feature of the studies included in this review was the low geographical representativeness and/or the low representativeness in the population studied. Conversely the analyses of large samples were not systematically stratified by subpopulations. We believe that such investigations need to zoom out (encompass a large population) and to zoom in (stratify by subpopulations) to get a complete picture evaluating the effectiveness of CIMPs.

To evaluate properly the PE of a CIMP we recommend to pay attention to the following points: (i) encompass all age groups and genders, except for targeted interventions such as IPTp or IPTi, (ii) sample all geographical and/or cultural patterns, (iii) stratify the evaluation of the PE by subgroups, (iv) adjust for socio-demographic variables that are associated with the outcomes and at least adjust for age and gender if the population sampled is not homogeneous in this regard.

Abbreviations

ACT: artemisinin-based combination therapy, CCS: case-control survey, CSS: cross-sectional survey, CIMP: control intervention for malaria prevention, IEC: information, education and communication, IPTi: intermittent preventive treatment for infants, IPTp: intermittent preventive treatment of pregnant women, IQR: interquartile range, ITN: insecticide-treated nets, IRS: indoor residual spraying, LLIN: long lasting insecticidal nets, NIBN: non-impregnated bed nets, RDT: rapid diagnostic tests, SMC: seasonal malaria chemoprevention; SP: sulfadoxine-pyrimethamine

Data availability

Dataset 1: Data underlying the results presented in this systematic review. DOI, 10.5256/f1000research.12952.d182415¹⁸⁹.