Keywords
Bloodstream infection, FilmArray, MicroScan, Resistance genes
Bloodstream infection, FilmArray, MicroScan, Resistance genes
The changes made to the article is change of accuracy to specificity as this was highlighted by the review.
See the authors' detailed response to the review by Josette Raymond
See the authors' detailed response to the review by Ciira Kiiyukia
Bacteremia accounts for a large number of hospital admissions and results in high morbidity and mortality1. In sub-Saharan Africa, there is limited information on bloodstream infections (BSI)2,3, which can be partly attributed to the paucity of studies conducted in developing countries lacking high throughput BSI diagnostic technology. The availability of such equipment would facilitate widespread BSI detection in patients, help close this critical knowledge gap, and potentially save lives4,5.
Blood culture and analytical profile index (API, BioMerieux) strip analysis has been the conventional standard for bacteremia diagnosis in many hospitals throughout the world, including Kenya6,7. Using this technique has been a challenge because it is a labor-intensive process that requires experienced laboratory technologists and has a reportedly lower level of accuracy than other techniques such as conventional cultures2–6. To address this issue, diagnostic platforms such as the BioFire FilmArray (FA) and MicroScan WalkAway 40 plus (MS) are now widely used in Europe and the United States2–8.
The FA is a sophisticated closed-automated polymerase chain reaction (PCR) system that utilizes specific commercial pouches such as FA blood culture ID panel to identify BSI causative agents and antimicrobial resistance markers within 1 hour of a positive blood culture2. This platform can identify 24 causative agents of BSI (eight gram-positives, 11 gram-negatives, and give yeast species) and three antimicrobial resistance genes: mecA for methicillin, vanA/B for vancomycin and blaKPC for carbapenem by nested multiplex PCR5–8. The MS system can provide identification of bacteria within 16 to 20 hours, phenotypic antimicrobial profile (AST) and extended-spectrum beta lactamase (ESBL) using negative combo panel type 66 for gram negatives and positive combo panel type 39 for gram positives9,10 and identification of yeast within 4 hours using a rapid yeast panel (Beckman Coulter, United States).
The use of conventional methods requires considerably long turnaround time (TAT) from 12–72 hours. In potentially life-threatening cases of BSI, the microbial cause of infection must be identified as quickly as possible to ensure proper treatment and management of the disease. The newly employed methods FA® and MS® have never been used before in Kisii and Homa Bay hospitals.
In this paper, we compared the accuracy, sensitivity, turnaround time (TAT) and cost of new technologies against the conventional API strip technique based on past data from blood culture isolates. Based on our findings, we will identify whether these automated platforms would be capable of providing a rapid diagnosis of BSI in Kenya hospitals.
Stored blood culture samples received from Kisii Teaching and Referral Hospital and Homa Bay County Referral hospital were analyzed at Microbiology Hub Kericho. These collection sites were selected because they do not have the capacity to identify BSI pathogens to the species level. Kisii Teaching and Referral Hospital is a public hospital located in Kitutu Chache Constituency, Kisii County. It is located in Kisii town Central Business, District Hospital Road, and serves a population of over 1 million. Homa Bay County Referral Hospital is a government health center located in Homa Bay Township Sub-location, Homa-Bay Location, Asego Division, and Rangwe Constituency in Homa Bay County. It has a population of over 1 million. The common diseases in these areas are malaria, upper respiratory tract infections, typhoid, pneumonia, tuberculosis and HIV11.
This study analyzed BSI isolates data tested using FA, MS technologies and culture followed by API strip analysis. A total of 152 isolates data (122 blood culture isolates and 30 control isolates were analyzed. The study focused on the identification of BSI, accuracy, sensitivity, TAT as well as the cost of the items. Isolate analysis was performed at the Microbiology Hub Kericho (MHK), a United States Army Medical Research Directorate-Africa (USAMRD-A) facility working in collaboration with Kenya Medical Research Institute (KEMRI). The MHK performs surveillance and clinical research diagnosis of enteric pathogens causing acute diarrhea across Kenya in addition to screening blood culture samples for BSI agents.
Formula n = Z2 × P (1-P)/d2, n = Sample size, Prevalence (p) = 10.4%
95% confidence interval (z) = 1.96
Precision d2 = 0.05
n = 1.962 × 0.104(1-0.104)/0.052
Calculated sample size (n) = 143 isolates.
The prevalence was taken from the study by Maze et al. (2018)12 “The epidemiology of febrile illness in sub-Saharan Africa: implications for diagnosis and management”. The prevalence rate is based on East Africa data which is part of Kenya, the ranges in prevalence is between 10–20% in Kenya.
Blood samples were collected into BACTEC Plus Aerobic/F, Peds Plus Aerobic/F, Anaerobic/F and Lytic/10 Anaerobic/F vials (BD, United States) and incubated using the BacTec 9050 instrument (BD, United States) for 5 days to account for slow-growing pathogens. A positive signal was indicated by an increased fluorescence caused by the carbon dioxide released by an organism reacting with the vial dye. Positive blood culture samples were removed and processed to identify the organism. Samples were processed directly using the FA without need for prior subculture. As for the MS and the API strip method, samples were first gram-stained and sub-cultured on MacConkey agar, Blood agar plate, Sabouraud Dextrose agar, Hektoen enteric agar and Tryptic soy agar (Becton Dickson).
For identification by FA, samples were tested per the manufacturer’s instructions. First, the blood culture identification panel was inserted in the loading chamber, then hydration solution was added to the sample and then the panel was placed into the FA. The results were checked after 1 hour.
Pure colonies were used for the MS identification procedure. An inoculum of 0.5 McFarland standard equivalents was prepared by selecting 1 to 3 discrete colonies from pure culture on MacConkey agar (MAC) or blood agar plate (BAP) or Sabouraud dextrose agar (SDA) and suspended in 3 ml of the MS inoculum water (Beckman Coulter). From the solution, 100 μl was transferred and mixed with 25 ml of the MS inoculum water with pluronic and then poured into the sterile inoculator D set tray. The solution (140 μl) was transferred into a gram-negative or a gram-positive panel (Beckman Coulter), and then loaded into the MS and results checked after 18–24 hours and 4 hours for yeast organisms.
For manual biochemical analysis method, the API strips (BioMerieux, United States) and media (MacConkey agar, blood agar, Hektoen enteric agar, triple sugar irons and Sabouraud dextrose agar plates) were brought to room temperature. After this, 5 mL of deionized water was added to the tray along with the strips. The API ampules or equivalent suspension medium was inoculated with a single colony. The inoculation of wells for gram negative, gram positive and yeast organisms was done as per manufacturer’s instruction. The incubation box was closed and incubated at 36°C for 18–24 hours for bacterial while for the yeast it was incubated at 29°C for 48 to 72 hours. A positive signal was indicated by a colorimetric change that was interpreted using the API guidelines.
Overall accuracy was calculated as the (number of individual isolate identified using evaluated technique) / (total number of same isolates identified) x 100%), sensitivity values and the 95% confidence intervals (CI) for both of these metrics were analyzed using the Graphpad Prism 8.2.1 software. The TAT was defined as the time taken from sample preparation to identification13. The average cost per samples included consumable reagents and disposable supplies and was defined as the total cost of each assay per sample run.
The indeterminate results were resolved by comparing the two techniques FA and MS, where the two techniques agreed was taken as a true positive, the discordant results were repeated using the API technique (gold standard).
Ethical review for this work was obtained from the Kenya Medical Research Institute Scientific and Ethical review Unit-Scientific Steering Committee (KEMRI SERU-SSC) #3686 and Walter Reed Army Institute Research (WRAIR) Institutional Review Boards (IRBs) #2513. Consent was not sought for this study since it was determined that there was no interaction with human subjects.
The overall accuracy and sensitivity of each platform is shown in Table 1. For the FA, the calculated specificity was 99.04% (95% CI, 96.59-99.88%). Out of the total 152 isolates identified, FA technology was able to correctly identify 150 isolates resulting in a calculated sensitivity of 98.68% (95% CI: 95.33-99.84%). Similar to the specificity computation for the FA instrument, the specificity of the MS instrument was determined based on the number of isolates correctly identified by the MS out of the total number of isolates correctly identified. This yielded a specificity of 98.56% (95% CI: 95.86-99.70%). The MS technology identified 149 true isolates, which produced a calculated sensitivity of 98.68% (95% CI: 95.30-99.84%). The API strip analysis identified 150 isolates with specificity of 99.04% (95% CI: 96.59-99.88%) with no significant difference in overall specificity to the FA and MS. The sensitivity of API method was 98.68% (95% CI: 95.33-99.84%).
The accuracy and sensitivity for the evaluated three techniques is shown above. The bold values listed are the calculated accuracy or sensitivity means for each method. Shown in parentheses are the 95% CI values.
Parameter Evaluated | API | FA | MS |
---|---|---|---|
Specificity | 99.04% (96.59 - 99.88%) | 99.04% (96.59 to 99.88%) | 98.56% (95.86 - 99.70%) |
Sensitivity | 98.68% (95.33% - 99.84%) | 98.68% (95.33% to 99.84%). | 98.68% (95.30% - 99.84%) |
Next, we decided to breakdown which microbes that each platform correctly identified (Table 2). The FA and API were able to identify 150/152 BSI pathogens. Interestingly, the FA platform was able to identify 4/4 of Enterobacter cloacae isolates as opposed to API technique, which detected only 2/4. However, the FA only recognized Staphylococcus at the genus level. In addition, the system could not characterize Streptococcus anginosus and Streptococcus bovis at the species level as these bacteria were not in its database, and therefore, were identified as Streptococcus spp. The FA identified Salmonella isolates as Enterobactericiae while the API and the MS were able to identify these isolates as Salmonella spp. The MS was able to accurately identify the presence of all other isolates similar to the API method except with Staphylococcus spp. (93.75%) and Acinetobacter baumanii (85.71%), while the API method was 100% accurate for both. In addition, the MS correctly identified all other Streptococcus isolates besides S. pneumoniae, whereas the FA and API standard had accuracies of 83.3% and 85.7%, respectively. Results of identification using each technique are available (see Underlying data14).
The accuracy broken down by microorganism was calculated for each platform. The total number of isolates identified followed by accuracy percentage of FA, MS and API is listed above. For the FA, the “n/a” indicates that the isolates could not be identified beyond the family level (Enterobactericeae). Those rows with a dash mark (-) indicate those identified at specific genus level (Salmonella) and not group level (Enterobactericeae).
We limited the running time for each platform to the length of a normal working day (8 hours) (Table 3). Under this condition, the FA could only run eight samples. We therefore established this limit to ensure an equal number of samples per run across all the techniques. The turnaround time (TAT) for the FA per sample run was 1 hour 6 minutes, with an average of 5 minutes processing time and 1 hour identification and results analysis. The average time for FA was 8 hours 40 minutes and significantly less (p < 0.0001) compared to the MS and API. The MS had a mean TAT of 42 hours per sample run, with 27 hours processing time and the mean TAT was significantly more (p < 0.0001) compared to the FA. Breaking this down, the culture process and incubation required 24 hours, which could be more depending on the bacteria and purity of culture. The API method had a mean TAT of 53 hours per sample run for bacterial isolates while for yeast was 103 Hrs. The mean TAT was significantly more (p < 0.0001) compared to the FA. Sample processing data are available for each technique (see Underlying data14).
For each platform being analyzed, turnaround times were broken into processing and ID analysis. The time shown for each of these categories is the mean of multiple runs, which had eight samples each. The processing column shows the time taken to prepare the samples for culture and then selection of isolates for identification. Of note, the FA did not require gram staining or culture and selection of an isolate from a blood culture. Identification analysis was based on the time each platform required to complete their respective protocols. The p-value showed the significance of the mean difference among the techniques.
Equipment | Processing time per 8 samples (min) | ID analysis per 8 samples (min) | Total time (min) | Total number of runs |
---|---|---|---|---|
FA | 40 | 480 | 520 | 19 |
MS | 1620 | 960 | 2580 | 19 |
API | 1620 | 1680 | 3300 | 19 |
The total average cost of processing one sample using the FA technique was 140.11 USD (Table 4). The total average cost of running one isolate per sample using the MS instrument was 38.75 USD while API technique was 29.17 USD. Extra equipment required but not included were biosafety cabinet, incubators, autoclaves, hot plates, conical flasks, stirrers and spatulas as well as the annual preventive maintenance for this equipment. Only the costs of the consumable items needed for each method were evaluated. As expected, the cost of the items used for the API technique was lower than for those used by the MS and FA.
The average cost of running sample was broken down into biochemical reagents/media used. Reagents not used by a platform as designated N/A (non-applicable). Not included in the cost breakdown were the equipment aforementioned above. The periodic maintenance required for the automated platforms was also not included.
In resource-limited settings, the use of conventional methods in diagnosis of bacteremia has been a challenge to most public health facilities leading to misclassification of the diagnosis of BSI8,15. The automated methods FA and MS proved to be more efficient, reliable and faster in the identification of a wide range of microorganisms than API. The technologies are reliable with a short turnaround time. These positive factors outweigh the use of API strips for microbial identification, which is considered the conventional standard in Kenya for diagnosis of BSI. In comparison to the FA and MS, the API method was more labor intensive. Furthermore, fastidious bacteria might not be identified if they fail to grow on culture media but can be identified directly from blood culture using FA.
Overall, the sensitivity of FA (98.68%), MS (98.68%) and API (98.68%) were identical, with an overall accuracy of 99.04%. Moreover, the sensitivity of FA demonstrated in this study was similar to the sensitivity observed in a previous study carried out in Kazulu-Natal16. The differences in sensitivity came in inability of FA and MS to agree in terms of genus and species individual identification of BSI pathogens.
The higher accuracy by FA in individual identification of BSI pathogens could be because it is a molecular-based platform. The FA identified Enterobacter cloacae with a higher accuracy than the other two methods, which could not identify the two isolates. This is probably because the FA identified two isolates as Enterobacter cloacae complex, which neither of the other methods could identify. However, the accuracy of FA was limited when identifying Streptococcus spp. where the accuracy was 83.3%, the organisms in question (Streptococcus anginosus and Streptococcus bovis) are not available in the FA database and are not common causes of BSI though they take advantage of immunocompromised individual and cause endocarditis2. The overall accuracy demonstrated by the FA is in line with previous paper evaluating the diagnostic capabilities of the system17.
The MS was able to accurately identify the presence of BSI bacteria with similar accuracy to the API method, except for the identification of Staphylococcus spp. where the accuracy was 93.75% and Acinetobacter baumanii with an accuracy of 85.71% for MS. The MS surpassed the API strip method in the identification of Streptococcus spp. where the accuracy was (100%) compared to API method (85.71%). These issues with MS in identifying Staphylococcus spp. and Acinetobacter baumanii is in line with previous studies, where the MS misidentified Acinetobacter baumanii18,19. For this study, the misidentified Streptococcus spp. were actually Staphylococcus spp.
In past studies, the API strip analysis had a lower accuracy identifying microorganisms such as Citrobacter species, Escherichia coli, Pseudomonas aeruginosa and Enterobacter species than automated platforms20. Interestingly, this did not occur with this study, and could partly be because the lab technician performing the assay had extensive clinical microbiology experience. Microbiology labs typically address this lower accuracy by adding biochemical tests such as oxidase and catalase to increase accuracy. We, however, did not incorporate these assays into the API strip analysis.
The mean TAT difference per run of eight samples among the technologies was significant at p<0.0001. The FA technology required 8 hours 48 minutes per eight samples compared to the MS, which required 42 hours and the API method, which required 53 hours for bacterial species and 103 hours for yeast. The major factor contributing this difference was time needed to prepare the isolates, which require gram-staining then culturing for 24/48 hours prior to identification by MS or API20. It should be noted that the MS has higher throughput and can process 40 or more panels in one run. In addition, the API method can test more samples and is only dependent on availability of incubators, reagents and the experience of the technician. The shorter TAT for FA is a very attractive feature for under-developed areas with poor infrastructure and inaccessible areas where field clinical/research activities are undertaken and do not necessarily require a high-throughput machine. Though not a metric evaluated in this study, the FA requires considerably less training and skill compared to the other methods, which help to balance its throughput limitations.
The average cost of testing one sample using FA was noticeably higher than the cost of the MS and API methods. This was expected as the FA test kits cost more than the MS panels and the API reagents. While the FA is more expensive, it is able to identify co-infection in one sample, which would require separate runs for the MS and API5.
Of note, the FA is able to identify resistant genes such as methicillin resistance common with Staphylococcus aureus, vancomycin resistance common with Enterococcus spp. and carbapenem resistance common with Klebsiella pneumoniae and other Enterobactericiae8. While the MS has no capability to identify antimicrobial resistant genes commonly associated with BSI, it is able to perform phenotypic drug sensitivity. In fact, the MS has a wider range of antimicrobial testing capabilities with regularly updated software database in line with CLSI guidelines.
While the evaluated methods were similar in accuracy and sensitivity, there were appreciable differences in TAT and cost. The FA cost more, but had a quicker TAT compared to the MS and API methods. This is a significant concern when using the machine in areas with limited financial resources. However, the FA requires minimal training prior to use and is able to identify co-infections. Furthermore, the FA requires a small space, and therefore, the cost of the FA panels should not be considered a major drawback since early detection of BSI has shown to reduce medical costs, hospital stays, and help guide the clinicians on the best treatment approach5, which lower overall economic costs.
The FA and MS have not been evaluated at Kenya hospitals and further evaluation using a larger sample size is recommended in order to have more data on BSI pathogens and their antimicrobial susceptibilities in different localities. However, these preliminary results clearly suggest that both the FA and MS platforms are valuable tools in rapid identification of BSI. Each technology has its advantages and disadvantages, which must be considered. Still, implementation of either platform could result in reduction of hospital stays, lower cost, better patient management and more appropriate use of antibiotics by clinicians.
Figshare: IDENTI~1.DOC. https://doi.org/10.6084/m9.figshare.12948533.v414.
This project contains the following underlying data:
Identification of Selected Primary Bloodstream Infection Pathogens in Patients Attending Kisii Level Five and Homa Bay County Hospitals- FA.xlsx. (Data obtained using Film Array.)
Identification of Selected Primary Bloodstream Infection Pathogens in Patients Attending Kisii Level Five and Homa Bay County Hospitals-MS.xlsx. (Data obtained using Microscan.)
Identification of Selected Primary Bloodstream Infection Pathogens in Patients Attending Kisii Level Five and Homa Bay County Hospitals-Api.xlsx. (Data obtained using analytical profile index.)
Identification of Selected Primary Bloodstream Infection Pathogens in Patients Attending Kisii Level Five and Homa Bay County Hospitals-FA raw data.xlsx. (Pathogen count data obtained using Film Array.)
Identification of Selected Primary Bloodstream Infection Pathogens in Patients Attending Kisii Level Five and Homa Bay County Hospitals-MS raw data.xlsx. (Pathogen count data obtained using Microscan.)
Identification of Selected Primary Bloodstream Infection Pathogens in Patients Attending Kisii Level Five and Homa Bay County Hospitals-Api raw data.xlsx. (Pathogen count data obtained using analytical profile index.)
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
We acknowledge the role of United States Army Medical Research Directorate-Africa, Kenya, the Microbiology Hub – Kericho, and Kenya Medical Research Institute for facilitating this study. Ethics committees from both WRAIR and KEMRI-SERU played a critical role in ensuring that this study is feasible and meets the required standards and invaluable gratitude also goes to Walter Reed Project-Kericho Regulatory office, Michael Obonyo, Mary Leelgo and Judith Bosuben for unrelenting support in their guidance, our thanks also goes to Benson Singa for the support during regulatory process and Rukia Kibaya for continued review and guidance.
Ronald Kirera was responsible for conceptualization, study design, data analysis, and drafting of the manuscript. Daniel Kariuki, Joseph Nganga, Judd L. Walson, Christine Hulseberg and Alexander Flynn contributed in the manuscript review. Elizabeth Odundo, Erick Kipkirui, Cliff Odhiambo, Nancy Kipkemoi, Abigael Ombogo, Janet Ndonye, Mary Kirui, and Margaret Koech contributed in data analysis and manuscript review.
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Infection and Drug Resistance
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Partly
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Infectious diseases (pediatric mainly) andd Helicobacter infection
Is the work clearly and accurately presented and does it cite the current literature?
Yes
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Medical microbiology-Bacteriology
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |||
---|---|---|---|
1 | 2 | 3 | |
Version 2 (revision) 28 Sep 22 |
read | ||
Version 1 12 Oct 20 |
read | read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)