Two parallel reporting systems for malaria surveillance in Pakistan , 2013 – 17 : is exact burden reflected ?

Pakistan is facing challenges regarding the availability of Background: reliable data for malaria surveillance. These include lack of coordination between different reporting systems and fragmented information system. This study aimed to compare the reporting of malaria surveillance systems in Pakistan. There are two parallel reporting systems for malaria surveillance Methods: in Pakistan, the District Health Information System (DHIS) and Malaria Information System (MIS). DHIS reports on all morbidity at health facility level, while MIS is only used for malaria surveillance in the donor supported districts. A cross sectional study was conducted between July-September 2018 by using the retrospective records of DHIS and MIS data reported to the Directorate of Malaria Control (DOMC) Islamabad during 2013-17. Descriptive and inferential analysis was performed to compare the coverage, outcome and impact indicators. During 2013-17, all districts (n=145, 100%) across Pakistan Results: reported on the DHIS. The MIS reporting coverage has gradually increased from 21 (14.5%) to 72 (49.7%) districts. Reported number of suspected screened and confirmed malaria cases were compared. MIS reported twice the number of suspects screened for malaria (100.5%) and confirmed malaria cases (124.4%) as compared to the DHIS. The difference in the reported average annual blood examination rate (ABER) was 3.8, test positivity rate (TPR) was -0.9 and the annual parasite incidence (API) was 4.9/1000 population over five years between two systems. DHIS reported only half the ABER and API as compared to MIS. There is huge under-reporting of suspected and confirmed Conclusion: malaria cases in the DHIS as compared to MIS. Urgent attention is needed to address this, as it is vital to have uniform reporting of true disease burden across the country. An integrated disease surveillance system, improved data validation systems, and use of the online DHIS-2 are potential options for better integrity and coherence of reported data.


Introduction
Malaria ranks sixth amongst the top ten causes of deaths in low income countries of the world 1 . In 2016, 91 countries reported an increase in malaria cases (216 million) as compared to 2015 (211 million). Around 0.4 million deaths have also been reported during the same year. Most cases were reported by the World Health Organization (WHO) African Region (90%), followed by the South-East Asia Region (7%), and the Eastern Mediterranean Region (2%) 2 . The incidence of malaria cases varies from low to high in different countries. Countries with low malaria incidence are progressing towards malaria elimination, while others having a high disease burden, including Pakistan, have implemented malaria control programs 3 .
The World Health Organization (WHO) has emphasized the critical need for transforming malaria surveillance as a core intervention in the Global Technical Strategy for Malaria. WHO has further stressed on the importance of prioritizing investments in malaria surveillance system to ensure that reliable data is available for decision making 4 . Effective surveillance of malaria is essential for identifying and prioritizing the most affected areas or population groups 5 . Moreover, uniformity of the surveillance tools and timeliness of reporting is important in countries with low disease burden at the malaria control phase 6 . It has been observed that despite all the efforts and investments, malaria surveillance system has many challenges related to the timeliness, representativeness, data quality and reliability in high and low burden countries from the WHO African and Eastern Mediterranean regions 7,8 .
Pakistan is among seven countries of the WHO Eastern Mediterranean Region sharing 95% of the regional malaria burden 9 . An estimated 98% of Pakistan population (205 million) is at varying risk, while around 60% population (123 million) at high risk for malaria 2 . In Pakistan, Malaria due to Plasmodium vivax is most common (88%), followed by Plasmodium falciparum (12%) 10 . Epidemiologically, Pakistan is classified as a moderate malaria endemic country with the national annual parasite incidence (API) averaging at 1.16, with a high variation within different provinces of Pakistan 11 . The districts and agencies in the Five provinces and Federally Administered Tribal Areas (FATA) region of Pakistan were stratified into three strata (I, II and III) based on the malaria annual parasite incidence (API), and slide positivity rate (SPR) of 2011-13 under the country's National Strategic Plan. Based on this stratification, 72 districts are placed in stratum-I (having API >5), ten in stratum-II (API 1-5) and 63 in stratum-III (API <1) for prioritizing the highest endemic districts for resource allocation 12 .
The malaria surveillance in the country has many issues. Major challenges include lack of coordination between different reporting systems, fragmented information systems and relying on parallel reporting for malaria cases in the highest burden sharing districts through a paper based malaria information system (MIS) and district health information system (DHIS) 13 . The diverse epidemiology of malaria disease in Pakistan stratifying the country into high and low burden sharing areas has further contributed to the difficulties in proper disease surveillance 12 .
It is estimated that due to such challenges, only around 23% of malaria cases have been captured in Pakistan through various surveillance systems during 2016 2 .
Limited evidence has been found regarding the comparison of various malaria surveillance systems in Pakistan. This study aims to compare the malaria coverage, screening, cases, outcome and impact as reported through the MIS and DHIS in high burden sharing districts of Pakistan.

Study design and setting
This was a cross sectional retrospective record review of malaria routine surveillance data for the period 2013-17 reported through the DHIS and MIS from donor supported districts of Pakistan. The study was conducted from July to September 2018 at the Directorate of Malaria Control (DOMC), Islamabad which is an attached department of the Ministry of National Health Services, Regulations and Coordination. DOMC is primarily responsible for malaria surveillance in Pakistan in collaboration with the provincial malaria control programs. The Global Fund (donor) is supporting the malaria control interventions in the highest burden sharing stratum-I districts located mainly in the provinces of Balochistan, Sindh, Khyber Pakhtunkhwa (KP) and FATA 12 .
Malaria surveillance data from the public health facilities is reported through two parallel systems, namely the DHIS (introduced in 2008 for all districts) and MIS (only for the Global Fund supported districts). DHIS reports on all morbidity at health facility level while MIS is used in only the donor supported districts for malaria surveillance 13 . Fever cases with signs and symptoms of malaria are screened as suspected cases, confirmed through the microscopy or rapid diagnostic test (RDT) kits, and then reported in the DHIS and MIS according to the identified species, i.e. P. falciparum, P. vivax or mix cases on monthly basis ( Figure 1). Data of five years as reported by the districts having both DHIS and MIS simultaneously from 2013 to 2017 was used for comparing the annual blood examination rate (ABER), test positivity rate (TPR) and annual parasite incidence (API).

Study population
All malaria cases reported to the Directorate of Malaria Control (DOMC) through DHIS and MIS between 1 st January 2013 and 31 st December 2017 from the districts where parallel reporting on both DHIS and MIS has been used.

Data collection, analysis and statistics
Epidemiological records of five years are available in electronic forms at the Directorate of Malaria Control in Islamabad. The data of this particular study was extracted from the DHIS and MIS. Study variables including the suspected cases screened for malaria, and confirmed malaria cases disaggregated by species (P. falciparum, P. vivax and mix) were doubled entered, and cleaned in the EpiData Entry version 4.4.3.1.
Descriptive analysis of selected variables such as reporting coverage of the two systems, annual blood examination rate, test positivity rates and reported annual malaria cases was performed in SPSS version 23. The DHIS and MIS data is regularly validated on monthly basis for the DHIS, and quarterly basis for the MIS at the district, provincial, and federal levels. Randomly, the hard copies of the reported data were matched with the entered data for validation.

Ethical approval
As this study was conducted on two malaria surveillance systems comprising of aggregated districts level data, there was no human subject directly involved in this study. Ethical and administrative approval (Reference F.No.2-30/2018/CMU-NFR; M&E/Surveillance/SORT-IT) was taken from the Director, DOMC for using the malaria program data for this study.

Results
During 2013-17, all districts (n=145, 100%) across Pakistan reported on DHIS. Reporting on the MIS increased gradually from 21 (14.5%) districts in 2013 to 72 (49.7%) districts in 2017. (Figure 2) For this study, data of only those districts was analyzed which had reported simultaneously on the DHIS and MIS during these five years. (Table 1) For screening the suspected malaria cases, DHIS reported a total of 4,260,610 suspected cases screened for malaria. During the same period, MIS reported a total of 8,540,702 suspected cases screened for malaria, which was 4,280,092 (100.5%) more than the DHIS from the same districts. The reported number of total confirmed malaria cases in the DHIS was 436,273. Out of these, P. vivax was the highest reported cause of malaria with 350,892 (80.4%), followed by P. falciparum (80,230, 18.4%) and mixed infection with 4,697 (1.1%). MIS reported 979,192 confirmed malaria cases during the same period for the same districts. P. vivax was the most reported cause of malaria (769,016, 78.5%), followed by P. falciparum (150,398, 15.4%) and mixed infection with 59,778 (6.1%). MIS reported 542,919 (124.4%) confirmed malaria cases more than the DHIS. (Table 2) DHIS reported an average annual blood examination rate (ABER) of 3.3, test positivity rate (TPR) of 13.6, and annual parasite incidence (API) of 4.4 for the five years. MIS reported an average ABER of 7.1, TPR of 12.7 and API of 9.3. The difference in the reported ABER was 3.8 (115.2%), TPR was -0.9 (6.6%) and API was 4.9 (111.4%). (Table 2)

Discussion
Pakistan like other low and middle-income countries (LMICs) has substantial reliance on external funding 14 . The Global Fund providing 50 percent of all international financing for malaria, has  Disease reporting in Pakistan across all provinces and at the federal level is carried out through the DHIS and several other parallel surveillance systems for the diseases according to the specific needs of the programs. Some of these parallel systems include Malaria, Dengue, TB, EPI, and HIV/AIDS. DHIS is the nation-wide health information system which is being used for reporting of the district level aggregated data from all the public health facilities 17 .
Our study shows that reporting coverage through the MIS, which is mainly used for reporting malaria in high burden districts of the country, has been increasing gradually from 21 to 72 districts from 2013-17. Balochistan and FATA provinces have the highest average number of districts reporting on both DHIS and MIS, which may be due to the highest annual parasite incidence in these provinces; hence they are supported by donor funds by the national program, and prioritized for malaria control interventions 18 . This study reveals major differences in the numbers of suspected screening for malaria and confirmed malaria cases reported between the two surveillance systems. A consistent under-reporting of suspected and confirmed malaria cases was seen in the DHIS as compared to MIS. This is contradictory to the findings of another recent study conducted in Swaziland comparing three reporting systems in malaria elimination settings where the national reporting system was found to be overreporting malaria cases 19 . Although both the DHIS and MIS are paper based surveillance systems, the difference in reported number of suspected cases screened and malaria confirmed cases observed from this study is very high. The reported confirmed malaria cases in MIS are 121.9% more as compared to the DHIS. Another study conducted in Malawi on data quality has shown discrepancy of 12-24% between paper-based and electronic data systems 20 . Ideally, there should be no variance between the two surveillance systems as the reported data is from same health facilities within the same districts.
The huge difference and under-reporting of malaria figures from DHIS may be due to several reasons. First, the start of donor support for malaria control interventions in the country has brought with it an additional reporting system to the DHIS, i.e., the MIS which is more comprehensive, and has many additional indicators as per the donor requirement. The MIS has its own data recording and reporting tools at the health facility level in parallel to already existing tools for DHIS. When there are parallel reporting systems with different recording and reporting tools in the same health facilities, the data quality is usually compromised as entering the data into separate platforms results in more errors 19 .
The second important factor to be explored will be that of who is responsible for data entry and reporting into DHIS and MIS at the very basic level, i.e. the health facility and district levels. Thirdly, there was considerable under-reporting of confirmed malaria cases in the DHIS. A possible reason may be that the tools in the DHIS are more specific for screening of malaria suspects with microscopy, which is the gold standard for confirmed diagnosis of malaria cases. However, DOMC with the support of the Global Fund grant, introduced RDT kits in health facilities without microscopy diagnosis facilities in the high burden districts, mostly in Basic Health Units (BHUs). This may have resulted in screening of more suspects using RDT at these health facilities and reporting of more confirmed cases. The cases screened and confirmed through RDTs may not have been reported into the DHIS by many of these centers as RDT is a relatively new diagnostic method 21 .
The study strengths are that the national surveillance data reported to routine program settings was used for analysis which is likely to reflect the reality on the ground. All districts reporting simultaneously on DHIS and MIS over the span of five years were included in the study across all provinces of Pakistan. The data reported in the MIS has been used as the benchmark as this database is carefully supervised and validated at the district level on a quarterly basis. The Global Fund grant has comprehensive data validation and quality assurance mechanisms in place hence we believe that the MIS data is more reliable 22 .
The current study is limited by the fact we did not have facility level data for analysis and we did not explore the exact reasons for observed discrepancies between the two surveillance systems. In-depth interviews of the district level supervisors and data entry operators for the DHIS and MIS along with the health facility staff actively involved in reporting on the two systems may help in better understanding of the dynamics and reasons for these discrepancies.
Despite of these limitations, the findings have a number of policy and practice implications. In the context of devolution in Pakistan, DHIS has utmost importance for timely surveillance of communicable diseases at the provincial level. MIS, being donor driven, is present in only around half of the country. The major concern in regards of the under-reporting seen in the DHIS is that the true malaria burden may be under-estimated. This under-reporting of the confirmed malaria cases can delay the early detection of malaria outbreaks predisposing the population to malaria epidemic. Moreover, the data from DHIS is used for decision making, disease prioritization and resource allocation according to the reported disease burden in the provinces. Under-reporting from DHIS will result in reporting of false burden of malaria cases, i.e. fewer cases than actual. This may result in lesser disease prioritization due to false reporting of low disease burden and hence less resource allocation.
Further research is required at the health facility, district and provincial levels to assess whether there is any mechanism for data validation, combined reporting, comparison and coherence for the DHIS and MIS data before being finalized for reporting into the two parallel systems. This is important as continuous validation of the health facility data is important before reporting into various systems 23 . The DHIS-2 being a free and open source, web based software may be the potential option for improving the completeness and quality of surveillance data being reported for malaria surveillance 24,25 .

ICMR-National Institute of Malaria Research, Jabalpur, Madhya Pradesh, India
Record of malaria cases was reviewed for the period of 2013-2017 retrospectively. Therefore the statement "a cross sectional study conducted between July -September" is confusing and need to be deleted.
Descriptive analysis to compare the coverage and reported suspected and confirmed malaria cases either by microscopy or by RDT was performed. No any inferential statistics was performed to analyze the relation between outcome and impact indicators. Therefore, this line should be corrected.
Authors did not mentioned the methods of surveillance for malaria screening in DHIS and MIS reporting system. As per our understanding DHIS is a passive and MIS is an active surveillance system. Patients with any ailments visited to the health facilities were reported in DHIS if they have febrile illness (Passive) and under MIS, specific field staff particularly engaged for malaria survey visited to the villages/households (door to door visit) for screening of suspected malaria cases (Active) and if it is true then analysis, discussion and conclusion should be revised accordingly. The previously published literature (Singh et al. 2016) 1 mentioned that less cases reported in passive surveillance as compared to active surveillance system.
Population coverage during the reported period is not mentioned whereas API and ABER indices are dependent with population coverage.  they attribute to the comprehensive data validation and quality assurance mechanisms embedded into Global Fund grants. They conclude that further research is required, but it is seems clear that the MIS platform is advantageous. Rather than conducting further research, what is likely to be more useful now are developing more detailed recommendations on lessons learned from the MIS system and how this could be adopted or incorporated into national platforms once donor funding ends.
Specific areas that would benefit from additional clarification or modifications in data presentation include:

Introduction:
"Stratifying the country into high and low burden sharing areas has further contributed to the difficulties in proper disease surveillance." -Some explanation as to why and in what ways this has caused issues would be helpful.

Methods:
This study was conducted between July and September -was there any influence of the season on the survey? ○ "Fever cases with sign and symptoms of malaria area screen as suspected cases, confirmed through microscopy and rapid diagnostic test (RDT) kits." -Is there any prior evidence of discordance or different reporting rates between the two methods in the country and how might this be accounted for in the analysis?

Results:
Is the reason that MIS reporting coverage has gradually increased due to the fact that the MIS is targeted and this indicates that burden has increased in these districts? Some details on how Global Fund MIS-supported districts are selected should be included. For instance, what are the risk-strata used to select?   The methods used to calculated ABER and TPR should be detailed in the footnotes, as well.

○
There should be further details in the methods regarding how cases are confirmed.
○ Percent difference is a somewhat unclear way to present the differences, as their comparison between different metrics is not easily interpreted.
○ ○ of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com