ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article

Enhancement of pyocyanin production by subinhibitory concentration of royal jelly in Pseudomonas aeruginosa

[version 1; peer review: 1 approved with reservations, 1 not approved]
PUBLISHED 11 Jan 2021
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

This article is included in the Cell & Molecular Biology gateway.

This article is included in the Pathogens gateway.

Abstract

Background: Pseudomonas aeruginosa, a multidrug resistant Gram-negative bacterium, produces pyocyanin, a virulence factor associated with antibiotic tolerance. High concentrations of royal jelly have an antibacterial effect, which may have the potential to overcome antibacterial resistance. However, in some cases, antibiotic tolerance can occur due to prolonged stress of low-dose antibacterial agents. This study aimed to investigate the effect of subinhibitory concentrations of royal jelly on bacterial growth and pyocyanin production of P. aeruginosa.
Methods: Pseudomonas aeruginosa ATCC® 10145â„¢ and clinical isolates were cultured in BHI media for 18 hours followed by optical density measurements at 600 nm wavelength to determine minimum inhibitory concentration (MIC). After 36 hours of incubation, pyocyanin production was observed by measuring the absorbance at 690 nm. Pyocyanin concentrations were calculated using extinction coefficient 4310 M-1cm-1.
Results: Results of the MIC tests of both strains were 25%. The highest production of pyocyanin was observed in the subinhibitory concentration group 6.25%, which gradually decreased along with the decrease of royal jelly concentration. Results of one-way ANOVA tests differed significantly in pyocyanin production of the two strains between the royal jelly groups. Tukey HSD test showed concentrations of 12.5%, 6.25%, and 3.125% significantly increased pyocyanin production of ATCC® 10145â„¢, and the concentrations of 12.5% and 6.25% significantly increased production of the clinical isolates.
Conclusions: This study concluded royal jelly concentrations of 25% or above could inhibit bacterial growth; however, only the concentrations of 12.5% and 6.25% could increase pyocyanin production in P. aeruginosa, both in ATCC® 10145â„¢ and clinical isolates. In conclusion, it is advisable to determine the appropriate concentration of royal jelly to obtain beneficial virulence inhibiting activity.

Keywords

royal jelly, antibacterial effect, Pseudomonas aeruginosa, pyocyanin

Introduction

Pseudomonas aeruginosa (P. aeruginosa) is one of the Gram-negative bacilli bacteria which causes nosocomial infections that can be fatal, especially in immunocompromised patients1–3. These bacteria are often found in the dental unit waterlines which allows the transmission of these bacteria into the oral cavity4. As an opportunist pathogen, P. aeruginosa is also frequently involved in oral infections, such as necrotizing ulcerative gingivitis, periodontitis, and mandibular osteomyelitis5–7. Although the mechanism is not clear yet, its presence in the oral cavity has been shown to result in systemic infections, such as nosocomial pneumonia8.

Based on the reports from several clinical cases, the infection caused by P. aeruginosa bacteria can be fatal. Treatment of P. aeruginosa infection is sometimes ineffective, which is closely related to the number of virulence factors possessed by the bacteria9. The bacterial cell surface components and some secretory products are important virulence factors of P. aeruginosa, one of which is pyocyanin10. Pyocyanin is a cytotoxic pigment from the Phenazine group of compounds that can facilitate biofilm development, cause pro-inflammatory effects, and result in host cell death11.

The resistance of P. aeruginosa to various spectrums of antibiotics creates difficulties in handling the infection it causes12. It has been reported recently that the administration of antibiotics below the minimum inhibitory concentration (MIC) can cause specific bacterial responses, such as an increase in pyocyanin production in P. aeruginosa. PAO1 and P14 are the attempts by the bacteria to survive under antibiotic stress13. This certainly motivates researchers to further analyze the infection they cause, and find the appropriate antibiotic concentration or dose to overcome the problem.

Royal jelly is a natural bee product that has the potential to be developed to overcome antibiotic resistance. Royal jelly has anti-inflammatory, antibacterial, and antioxidant effects14. Royal jelly proteins, such as Jelleine, major royal jelly protein-1 (MRJP1), and royalicin are known to have antibacterial effects against P. aeruginosa. Major royal jelly protein-1 and Jelleine can interfere with the permeability of the outer membrane of the cell, causing the loss of vital contents of bacterial cells, which in turn causes cell death. Cationic antimicrobial peptides, such as royalicin, are known to also interfere with cell membrane permeability in various Gram-positive and Gram-negative bacteria, such as P. aeruginosa15–17. Results of previous studies have shown that royal jelly can inhibit the growth of P. aeruginosa. In this study, royal jelly showed inhibition of the growth of P. aeruginosa ATCC® 27853™18. In addition, it has also been known that royal jelly in various concentrations can inhibit the nonspecific attachment of P. aeruginosa ATCC® 27853™19, but so far, the effect of the subinhibitory concentration of royal jelly against these bacteria is unknown. Furthermore, as pyocyanin is an indicator of the pathogenicity of P. aeruginosa strains, the aim of this study was to determine the effect of subinhibitory royal jelly concentration on pyocyanin production in representative strains of a high level pyocyanin-producer P. aeruginosa ATCC® 10145™ and clinical isolates.

Methods

This in vitro laboratory experimental research was done at the Integrated Research Laboratory of the Faculty of Dentistry, Universitas Gadjah Mada, Yogyakarta. All research procedures have been approved by the Ethics Committee of the Faculty of Dentistry, Universitas Gadjah Mada, Yogyakarta (No. 00393/KKEP/ FKG-UGM/EC/2020).

The royal jelly used in this study was obtained from Nusukan, Surakarta, Central Java, Indonesia. This product is produced from Apis mellifera bees that have been identified previously19. Royal jelly 5.5 grams was dissolved in 10 ml of cold phosphate buffered saline (PBS), then homogenized using a magnetic stirrer (24 hours, 4°C). The royal jelly solution was centrifuged (12,000 g, 45 minutes, 4°C), then the supernatant was filtered using 0.45 µm millipore to produce 55% royal jelly. Furthermore, royal jelly was stored at a temperature of 4–8°C20.

Pseudomonas aeruginosa ATCC® 10145™ (Thermo Scientific) was obtained from the Integrated Research Laboratory of the Faculty of Dentistry, Universitas Gadjah Mada. A clinical isolate of P. aeruginosa derived from patient sputum was obtained from the Laboratory of Microbiology, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada. Both of these strains were each inoculated in Luria Bertani broth and incubated at 37°C for 24 hours. After that, the culture was centrifuged at 3000 rpm for 15 minutes and then resuspended using 0.98% NaCl to obtain a bacterial concentration equivalent to 1.5 × 105 CFU/ml.

Measurement of the effect of royal jelly on the viability of P. aeruginosa

A sterile 55% w/v royal jelly solution was diluted in brain heart infusion (BHI; Himedia Laboratories) broth to obtain a concentration of 50% and then serial dilution was performed in 96 well microplates. A total of 5 µl of the P. aeruginosa ATCC® 10145™ suspension or clinical isolate bacteria (1.5 × 105 CFU/ml) was inoculated in all groups, except the groups that had been determined as blanks (blanko). The culture was then incubated at 37°C for 18 hours. After that, the microplate was scanned using the Spark® Multimode Microplate Reader (Tecan trading AG) to measure optical density (OD) using a 600 nm wavelength. The percentage of bacterial viability inhibition was determined based on the OD value of the treatment group against the control.

Analysis of the effect of royal jelly on pyocyanin production

Royal jelly solution was diluted into sterile BHI broth to get the concentration of 12.5%, 6.25%, 3.125%, 1.56%, 0.78%, 0.39%, 0.19%, and 0.098% w/v. Both strains of P. aeruginosa were cultured on BHI broth containing various concentrations of royal jelly as treatment and BHI broth only as a blank (blanko). The cultures were incubated at 37°C for 36 hours, then the pyocyanin production of each strain was observed visually, which appeared green in the culture supernatant. The pyocyanin concentration was further quantified using previously published methods21. Briefly, after 36 hours of incubation, the culture supernatant was transferred to a sterile tube and centrifuged at a rate of 10,000 g for 30 minutes. The supernatant was filtered using a 0.45 µm Millipore filter and transferred to 96 new well microplates. The absorbance value of the supernatant containing pyocyanin was measured at a wavelength of 690 nm, then the pyocyanin concentration was calculated using the following equation21.

Concentration of pyocyanin = A690 nm (A690 nm of sample - A690 nm of blank) / ε * d

ε = extinction coefficient (pyocyanin at A690 nm = 4310 M-1cm-1)

d = path length (0.23 cm for 96 well microplate)

Statistical analysis

The data in this study were presented as the percentage of bacterial viability and pyocyanin concentration in the P. aeruginosa culture supernatant. All data were tested for normality using the Shapiro-Wilk and the Levene Test for homogeneity using SPSS Statistic v20. Furthermore, one-way ANOVA and Games-Howell parametric analysis were performed for bacterial cell viability data; and parametric one-way ANOVA followed by Tukey HSD on pyocyanin concentration data.

Results

Antibacterial activity of royal jelly against P. aeruginosa

The antibacterial activity of royal jelly against the two strains of P. aeruginosa is shown in Figure 1. Data on the percentage of bacterial growth inhibition shows normal distribution data (p>0.05), but has a non-homogeneous variant (p<0.05). One-way ANOVA showed a significant difference in the percentage of growth inhibition in P. aeruginosa ATCC® 10145™ (p = 0.000) and P. aeruginosa clinical isolate (p = 0.000) between royal jelly treatment groups and negative control. In this study, it was proven that royal jelly can inhibit the viability of both P. aeruginosa strains starting from a concentration of 25%. The results of the multi-comparison analysis showed that there was no significant difference between the concentrations of 25% and 50% and significant differences were identified between the concentrations of 25% and 50% with 12.5% to 0.098% in both strains. It can be concluded that the MIC for both strains is 25%.

f46febd9-1eac-4fa6-b13a-2c6f8f988ffc_figure1.gif

Figure 1. Percentage of inhibition of growth of P. aeruginosa bacteria (1.5 × 105 CFU/well).

Bacterial cultures were incubated with varying concentrations of royal jelly for 18 hours at 37°C. Royal jelly 50% and 25% inhibit bacterial growth. The difference is based on the results of the Games-Howell analysis at the significance value (*) p <0.05. ATCC® 10145™ (a) strain; clinical isolate (b).

Exposure to subinhibitory royal jelly concentrations induced increased pyocyanin production in P. aeruginosa

Pyocyanin was identified as green in culture supernatant P. aeruginosa ATCC®10145™ and clinical isolate. After 36 hours of incubation, pyocyanin production was increased in the stimulated culture group with subinhibitory concentrations below 25%. The intensity of green color in the culture medium increased with the increase in the concentration of royal jelly (Figure 2). The change in the color intensity of the culture supernatant was consistent with the results of the pyocyanin concentration measurement.

f46febd9-1eac-4fa6-b13a-2c6f8f988ffc_figure2.gif

Figure 2. Pyocyanin was identified from the green color of the P. aeruginosa culture supernatant after 36 h incubation at 37°C.

P. aeruginosa bacteria (1.5 × 105 CFU/well) ATCC® 10145™ (a); clinical isolate (b). Royal jelly 12.5% (A); 6.25% (B); 3,125% (C); 1.56% (D); 0.78% (E); 0.39% (F); 0.19% (G); 0.098% (H); 0% (I); No treatment (J).

Pyocyanin concentration data in each royal jelly treatment group and negative control were the results of experiments on triplicate cultures. Figure 3 shows the average pyocyanin concentration for each group. The highest average pyocyanin concentration was identified in P. aeruginosa ATCC® 10145™ induced by royal jelly with a concentration of 6.25%, which was 23.59 μM, while the lowest mean was identified in clinical isolates of P. aeruginosa without exposure to royal jelly, which was 0.7 μM. The pyocyanin concentration of P. aeruginosa ATCC® 10145™ was seen to be higher than clinical isolate in the same concentration in all treatment groups.

f46febd9-1eac-4fa6-b13a-2c6f8f988ffc_figure3.gif

Figure 3. Pyocyanin production in P. aeruginosa bacteria (1.5 × 105 CFU/well) after 36 hours of exposure to various concentrations of royal jelly.

Pyocyanin production increased due to exposure to sub-inhibitory royal jelly concentrations of 6.25% and decreased at lower royal jelly concentrations. The difference was based on the results of the Tukey-HSD analysis at the significance value (*) p <0.05. ATCC® 10145™ (a) strain; (b) clinical isolate.

Pyocyanin concentration data both on ATCC® 10145™ and clinical isolate in all groups were normally distributed (p>0.05) and homogeneous (p>0.05). There was a significant difference in the concentration of pyocyanin ATCC® 10145™ (p = 0.000) and clinical isolate (p = 0.000) between the treatment groups. The results of multiple comparison analysis of Tukey-HSD on P. aeruginosa cultures of ATCC® 10145™ showed a significant difference between the royal jelly groups with concentrations of 0% with 12.5%, 6.25%, and 3.125%. In addition, a significant difference in pyocyanin concentrations in clinical isolate was found between the 0% royal jelly group with 12.5% and 6.25%.

Discussion

The antibacterial effect of royal jelly has been widely reported by previous researchers15–17. The ability of royal jelly to inhibit the growth of P. aeruginosa is thought to be related to the variety and concentration of its antibacterial protein. Royal jelly components that have been identified as having antibacterial activity are major royal jelly protein-1 (MRJP-1), Jelleine I–III, royalicin, and 10-hydroxy-2-decenoic (10-HDA)15,17,22.

This study showed that royal jelly concentrations of 25% and 50% had antibacterial activity against P. aeruginosa ATCC® 10145™ and clinical isolate. The results of this observation are different from previous studies that showed P. aeruginosa growth could be inhibited at concentrations >50%18. This difference is thought to be closely related to differences in geographical location, botanical origin, climate, and storage conditions of royal jelly, which affect the antibacterial component of royal jelly23. Previous studies have shown that royal jelly originating from different geographic and botanical locations affects the quantity of 10-HDA. Royal jelly originating from tropical climates is reported to contain lower concentrations of 10-HDA than cold climates23. The higher temperature and longer storage time also resulted in a significant reduction in the quantity of MRJP124. However, the bacterial strains studied probably also had an effect, as previously reported there was a variable response between clinical isolates and standard bacteria18,19.

Pyocyanin is an indicator of the pathogenicity of P. aeruginosa. To our knowledge, this study report is the first to demonstrate a dualism effect of royal jelly on P. aeruginosa. The subinhibitory concentration of royal jelly amplify the effect of an autoinducer. It was able to increase the production of pyocyanin in ATCC® 10145™ and clinical isolates to protect and maintain their survival13. The pyocyanin concentration in the ATCC® 10145™ strain appeared to be significantly higher than the clinical isolates. This observation is in accordance with previous studies that found the ATCC® 10145™ strain produced more pyocyanin than the clinical isolate strains from active ulcerative keratitis patients25. The presence of phzM and phzS genes was thought to affect the concentration of pyocyanin produced26. This was proven by previous studies that the phzM and phzS gene expression of multidrug resistance (MDR) clinical isolate P. aeruginosa was lower leading to less pyocyanin production than non-MDR isolates and PAO1 strains27. Some clinical isolates were also reported not to have the genes so that these bacteria cannot produce pyocyanin26. Other research results also showed that the pyocyanin concentration of ATCC® 10145™ strains is higher than that of PAO1 and PA14 strains after incubation for 60 hours28. It is estimated that ATCC® 10145™ is one of the strong pyocyanin producing strains. However, other virulence factors possessed by this strain were lower than the clinical isolate strains so that they were considered less virulent25.

Various virulence factors, including pyocyanin are generally associated with the quorum sensing mechanism29. Quorum sensing refers to the communication process between microbial cells using autoinducer molecules30. One of the autoinducer molecules that plays an important role in the regulation of pyocyanin production is the pseudomonas quinolone signal (PQS). Mutation of the PQS gene results in reduced pyocyanin production31. When bacterial cells are exposed to exogenic stress, such as an antibacterial agent that can threaten their survival, the bacteria immediately respond to the stimulus by inducing the production of PQS which is responsible for activating various genes involved in the production of virulence factors, including pyocyanin29,32. Although the effect of royal jelly subinhibitor concentration on this autoinducer molecule is not yet known, several studies have reported that the increase in pyocyanin production is closely related to the effect of subinhibitor antibiotics that increase PQS gene expression33. It is thought that this is the cause of increased pyocyanin production at subinhibitory concentrations.

The increase in pyocyanin production in P. aeruginosa bacteria will have implications for the mechanism of bacterial attachment and biofilm formation. Apart from its production, which is closely related to the quorum sensing mechanism, pyocyanin is also a signaling factor in the quorum sensing process itself. This was identified from the results of research on P. aeruginosa PAO1 and PA1434. In addition, the increase in pyocyanin is likely to have an impact on the activity of bacteria to produce extracellular DNA (eDNA). Extracellular DNA is an important part of extracellular polymeric substance (EPS) which is the main component of the biofilm matrix. The increase in EPS production is very beneficial for the bacterial attachment process and subsequently the formation of biofilms. Pyocyanin can induce eDNA production in low level pyocyanin-producer strains, PAO1 and pyocyanin-deficient strains, PA14. In this study, it was proven that pyocyanin caused an increase in the production and release of eDNA, which is the main component in forming and stabilizing bacterial biofilms35.

The increase in pyocyanin production induced by subinhibitory royal jelly concentrations in P. aeruginosa ATCC® 10145™ and clinical isolates in this study is an interesting phenomenon. Although, subinhibitory royal jelly concentrations were not effective in inhibiting the growth of these bacteria, on the other hand, they increased production of pyocyanin virulence factors. This has inspired the alleged biphasic nature of royal jelly which has antibacterial potential, but at different exposure concentrations, it can induce the production of P. aeruginosa bacteria virulence factors. This phenomenon leads us to think that researchers, as well as medical practitioners, should be careful in determining the concentration of royal jelly for its antibacterial research purposes or its therapeutic potential. This of course requires further research on the mechanisms associated with bacterial response to subinhibitory concentrations of royal jelly.

Conclusions

Royal jelly at a concentration of 25% was only able to inhibit the growth of P. aeruginosa bacteria, but at subinhibitory concentrations it could increase pyocyanin production in P. aeruginosa strain ATCC® 10145™ and clinical isolate. Based on the results of this study, we suggest selecting the appropriate dose or concentration for the purpose of inhibiting the growth and production of P. aeruginosa virulence factors.

Data availability

Underlying data

Figshare: Pseudomonas aeruginosa pyocyanin, https://doi.org/10.6084/m9.figshare.13247429.v136.

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Comments on this article Comments (0)

Version 4
VERSION 4 PUBLISHED 11 Jan 2021
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Amly DA, Hajardhini P, Jonarta AL et al. Enhancement of pyocyanin production by subinhibitory concentration of royal jelly in Pseudomonas aeruginosa [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2021, 10:14 (https://doi.org/10.12688/f1000research.27915.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 1
VERSION 1
PUBLISHED 11 Jan 2021
Views
39
Cite
Reviewer Report 04 Feb 2021
Theerthankar Das, Department of Infectious Diseases and Immunology, School of Medical Science, The University of Sydney, Camperdown, NSW, Australia 
Not Approved
VIEWS 39
The intention of this study is good and to enlighten the antibacterial and anti-virulence effect of royal jelly in P. aeruginosa.

To be more conclusive this study needs more work:
  1. Its better to
... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Das T. Reviewer Report For: Enhancement of pyocyanin production by subinhibitory concentration of royal jelly in Pseudomonas aeruginosa [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2021, 10:14 (https://doi.org/10.5256/f1000research.30869.r77077)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
  • Author Response 13 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    13 Jul 2021
    Author Response
    We thank you for the thorough and careful review of our manuscript. We apologize for our delay in revisions due to the various obstacles we face regarding the availability of ... Continue reading
  • Author Response 09 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    09 Jul 2021
    Author Response
    We thank you for the thorough and careful review of our manuscript. 
    The following are our responses to the suggestions and specific comments.

    The response to comment 1 (quantification of ... Continue reading
COMMENTS ON THIS REPORT
  • Author Response 13 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    13 Jul 2021
    Author Response
    We thank you for the thorough and careful review of our manuscript. We apologize for our delay in revisions due to the various obstacles we face regarding the availability of ... Continue reading
  • Author Response 09 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    09 Jul 2021
    Author Response
    We thank you for the thorough and careful review of our manuscript. 
    The following are our responses to the suggestions and specific comments.

    The response to comment 1 (quantification of ... Continue reading
Views
36
Cite
Reviewer Report 01 Feb 2021
Ariadna A. Djais, Department of Oral Biology, Faculty of Dentistry, University of Indonesia, Jakarta, Indonesia 
Approved with Reservations
VIEWS 36
This study examined royal jelly and Apis mellifera bees as a base ingredient. These ingredients are antibacterial with pycocyanin as a pigment by the bacterium Psedomonas aeroginosa. The P. aeroginosa was selected because these bacteria are known as multidrug-resistant bacterium. This research is interesting and ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Djais AA. Reviewer Report For: Enhancement of pyocyanin production by subinhibitory concentration of royal jelly in Pseudomonas aeruginosa [version 1; peer review: 1 approved with reservations, 1 not approved]. F1000Research 2021, 10:14 (https://doi.org/10.5256/f1000research.30869.r77076)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
  • Author Response 13 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    13 Jul 2021
    Author Response
    Thank you for providing a review and suggestions for improving our manuscript. Below are our revisions to the manuscript according to the review comments.

    1. Response to comments 1:
    ... Continue reading
  • Author Response 09 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    09 Jul 2021
    Author Response
    Thank you for providing a review and suggestions for improving our manuscript.
    Below are our revisions to the manuscript according to the review comments.

    1. The response to comment ... Continue reading
COMMENTS ON THIS REPORT
  • Author Response 13 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    13 Jul 2021
    Author Response
    Thank you for providing a review and suggestions for improving our manuscript. Below are our revisions to the manuscript according to the review comments.

    1. Response to comments 1:
    ... Continue reading
  • Author Response 09 Jul 2021
    Heni Susilowati, Department of Oral Biology, Faculty of Dentistry, Universitas Gadjah Mada, Sleman, 55281, Indonesia
    09 Jul 2021
    Author Response
    Thank you for providing a review and suggestions for improving our manuscript.
    Below are our revisions to the manuscript according to the review comments.

    1. The response to comment ... Continue reading

Comments on this article Comments (0)

Version 4
VERSION 4 PUBLISHED 11 Jan 2021
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.