Topical Nasal Steroids for Allergic Rhinitis in Children. Is One Better Than the Others? Protocol for a Systematic Review and Network Meta-analysis.

Study design and registration

We present a protocol for a systematic review and network meta-analysis registered in the international prospective register of systematic reviews, PROSPERO (Register number 476313), and following the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) guidelines.^19,20 The current protocol has not undergone amendments thus far. Should any amendments be required, they will be reported and dated in PROSPERO along with the reason for such modification. We declare that this research project does not have institutional sponsorships.

Data sources and search strategies

We will conduct searches to identify clinical trials without language restrictions, publication status limitations, and without date restrictions up to the time of the search. The sources will include MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials (CENTRAL), the Latin American and Caribbean Health Sciences Literature Database (LILACS), and Epistemonikos. Additionally, we will explore sources of grey literature such as Clinicaltrials.gov, the World Health Organization International Clinical Trials Registry Platform (ICTRP), The National Institutes of Health database of funded studies for ongoing or unpublished trials, presentations at international congresses in otolaryngology, allergology, or pediatrics, and targeted searches on pharmaceutical company websites. We will set up monthly notification alerts for new eligible articles, and search strategies will be rerun prior to article submission for publication. An expert medical librarian will develop the literature searches. In the extended data section we provide the details of the search strategy.

Eligibility criteria for studies

Randomized clinical trials, whether blinded or not, parallel or crossover in their first phase (to prevent contamination due to insufficient “washout” effect), comparing TNS against each other or placebo or no treatment, will be eligible. The evaluation will focus on efficacy and safety in children up to 16 years of age with a clinical diagnosis of moderate to severe AR. The definition of moderate/severe AR will follow the criteria outlined by the ARIA consensus: the presence of nasal obstruction, nasal itching, sneezing, and rhinorrhea, which can be intermittent or persistent and affect daily life activities or be associated with asthma.^2,6–8,21 According to ARIA, no laboratory tests are necessary for diagnosis.^2,21 Studies that recruit both children and adults may be included if the two populations are analyzed separately. When the study does not separately report the populations, we will contact the study authors to obtain separate results, where possible.

Studies allowing co-intervention with oral antihistamines or mast cell stabilizers will be included if these co-interventions were considered from the protocol stage and were administered similarly across the trial arms. Studies permitting co-interventions with systemic steroids, leukotriene antagonists, antihistamines in nasal spray form, immunotherapies, or alternative treatments will not be eligible.

Our primary outcome is the treatments’ efficacy measured as change in the Total Nasal Symptom Severity Score (TNSS) between baseline and the final measurement. The TNSS includes the following items: nasal obstruction, rhinorrhea, sneezing, and nasal itching, each graded from 0 (absence of the symptom) to 3 (severe symptom interfering with daily life). The sum of all symptoms grades the severity of the disease from “0” asymptomatic to “12” severely symptomatic in all aspects.^22,23 Standardized mean differences will be used if different scores or scales are used to measure symptoms severity. We will prefer studies that report the change in the score, but in case there is not enough information, we will analyze the mean difference of the final scores. Measurements of the primary outcome performed between 3 weeks and 6 months and those made from 6 months onwards will be recorded separately. When a study includes repeated measurements, the mean of the measurements for each period will be analyzed separately in the categories detailed before.

Our secondary outcomes are Quality of life (QoL), adverse events (AE), and growth velocity. We will prefer QoL scales specific for AR. Thus, we will prioritize result presented using the Pediatric Rhinoconjunctivitis Quality of Life Questionnaire (PRQLQ)²⁴ and the Adolescent Rhinoconjunctivitis Quality of Life Questionnaire (ARQLQ).²⁵ However, global QoL measures will also be considered. We will record the change in baseline QoL scores compared to the final scores, using standardized mean differences as a summary measure.

Regarding AEs, we will record both the number of AEs for each intervention and the proportion of patients experiencing at least one AE. We will use the definition provided by the Pan American Health Organization, which classifies AEs as Fatal (contribute directly or indirectly to the patient’s death), Serious (life-threatening), Moderate (interfere with usual activities without directly threatening the patient’s life), and Mild (easily tolerated signs and symptoms not requiring treatment).^26,27

Lastly, for growth velocity we will focus on the annual gain in height measured by stadiometry (the child’s length from heel to crown with the patient standing). However, we are aware that many trials use knemometry (heel-to-knee measurement), which is used to measure short-term changes, despite its limited extrapolation to annual growth measured by stadiometry.^28,29 When growth velocity is measured using knemometry, we will use standardized mean differences, as an effect measure, and conduct sensitivity analyses for both types of measurements.

Study selection

We will follow the guidelines of the PRISMA 2020 collaboration³⁰ to create the study selection flowchart as presented in Appendix 2. The obtained references will be exported to the reference manager Zotero.³¹ After removing duplicates, the references will be independently and in duplicate by two reviewers (OS, VS, GS, LM) screened by the reviewers using the Covidence web-based software.³² We will use a title and abstract screening form to ensure that the article is a clinical trial evaluating a TNS for the management of AR in children. Before the formal screening process, we will conduct a pilot exercise.

The references considered eligible by at least one of the reviewers at the title and abstracts stage will be searched in their full version. Two reviewers will review the full texts of the eligible studies independently and in duplicate (OS, VS, GS, LM). We will include the studies that are considered eligible at this stage by both reviewers. Disagreements will be resolved by consensus or by the participation of a third reviewer (JY, IF).

Data extraction

Data extraction will be conducted independently and in duplicate by pairs of reviewers (OS, VS, GS, LM) using an online format designed in Covidence (Appendix 3). We will extract the following information: Author and year of the study, study design, type of analysis (per protocol PP or intention-to-treat ITT), molecule and dosage of each intervention, patient characteristics (average age and age range, sex, and type of AR), intervention duration, baseline, final, and change in nasal symptom scores and QoL scores, number and type of adverse events (AEs), average initial height, annual height gain, and monthly growth. In case of crossover studies, data from the first period will be extracted to avoid possible carryover effects.

In cases of missing data, we will contact the author via email (up to three times) to request the unavailable information. When data from outcomes is presented exclusively in figures, we will use the WebPlotDigitizer program (https://automeris.io/WebPlotDigitizer/) to extract the data.³³ We will conduct a pilot test with 10 studies before starting data extraction to minimize discrepancies between reviewers and make necessary adjustments to the extraction form. Disagreements will be resolved through consensus, and in case consensus cannot be reached, a third reviewer (IF; JY) will adjudicate the disagreement. If the data remains unclear despite these efforts, the outcome will be excluded from the analysis.

Direct comparisons and assessment of heterogeneity

We will present the outcomes in tables and provide a narrative description of the results. When possible, we will combine studies in pairwise meta-analysis per intervention comparison. As we anticipate variability in estimates due to potential methodological differences between studies and patient characteristics, we will use a frequentist random-effects model.³⁴ We will estimate mean differences with their respective 95% confidence intervals (95% CI) for continuous outcomes and relative risks (RR) for dichotomous data. We will estimate the overall effect and its 95% CI using the Hartung-Knapp-Sidik-Jonkman (HKSJ) method, according to the recent recommendations by Cochrane.^35,36 Along with 95% CIs for the overall effect we will calculate prediction intervals (95% PIs) to obtain an interval within which we expect a true intervention effect of a new study to lie within. To interpret the results, we will use the Minimally Important Difference (MID) of 0.55 points for TNSS and 0.4 points on the mini RQLQ scale, following Barnes et al.³⁷ If continuous data are not reported on the same scale, we will use the standardized mean difference to report the change of scores. For binary outcomes, we will add 0.5 in studies with zero-event arms, and if both arms lack events, the study will be excluded.³⁸

Assessment of heterogeneity

We will use the restricted maximum likelihood method to estimate the heterogeneity variance (τ²) and the Q-profile method to infer on its uncertainty (95%CI).³⁹ We will compare heterogeneity estimates with the empirical distribution proposed by Turner⁴⁰ and Rhodes⁴¹ for binary (using OR) and continuous data (using SMD), respectively. We will quantify the percentage of variability due to heterogeneity rather than sampling error using the I² statistic.⁴² We will interpret I² according to the Cochrane thresholds⁴³: (0% to 40%: not important; 30% to 60%: moderate heterogeneity; 50% to 90%: substantial heterogeneity; 75% to 100%: considerable heterogeneity).

As potential sources of heterogeneity, we a priori propose the administered dose⁷ and treatment regimen (once or twice daily) since evening doses of steroids more significantly suppress the Hypothalamic-Pituitary-Adrenal (HPA) axis^18,44; co-intervention with oral antihistamines, as it was the reason that prevented Al Sayyad from combining studies found until 2007¹²; the type of AR (intermittent or persistent)⁴⁵; private sponsorship^43,46; the time of measurements (between 3 weeks and 6 months and those made from 6 months onwards); and the risk of bias.^46,47 We will conduct subgroup analyses based on the previous effect modifiers and sensitivity analyses based on the RoB.

Regarding age, if data allows, we will try to analyze three groups: under 6 years, 6 to 12 years, and 12 or older, given their different susceptibility to growth velocity impairment by steroids.¹⁸ We will conduct sensitivity analyses according to the risk of biases (excluding the studies with high RoB)⁴⁸ and the need for data transformation or estimations due to missing data or data extracted from figures.⁴⁹ We will assess the credibility of subgroup analysis according to the criteria of the Instrument for Credibility Assessment for Outcome Modification Analysis (ICEMAN).⁴⁷ We will interpret the results of heterogeneity using the GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) approach.⁵⁰

Risk of bias assessment

We will assess the risk of bias for each included study independently by two reviewers (SO, SG) using the Cochrane Risk of Bias 2 (ROB2) tool.⁴⁸ This tool comprises 5 domains: bias arising from the randomization process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in outcome measurement, and bias in the selection of reported results. The tool allows for grading the risk in each domain and an overall assessment as high risk, some concerns, or low risk of bias. We will provide training to all reviewers on the use of the tool and conduct a pilot test to ensure proficiency. Disagreements in the risk level will be resolved through consensus, and in case of inability to reach consensus, a third reviewer (IF) will adjudicate.

Publication bias assessment

A funnel plot will be constructed for each direct comparison and outcome to assess the potential publication bias against small studies or those with negative results, provided there are at least 10 studies available.⁵¹

Network meta-analysis: direct and indirect evidence

We will conduct a random-effects NMA, as we expect substantial heterogeneity among studies. All analyses will be by intention to treat (i.e., include all randomized patients to any treatment arm). Several interventions may have not been directly compared head-to-head, resulting in limited direct evidence. In such cases, the estimated effect will be derived from indirect evidence. When direct evidence is available, the NMA will provide a network estimate (combining direct and indirect estimates).⁵² We will combine and obtain network estimates if assumptions of study homogeneity, and transitivity across direct comparisons are justified (44). Transitivity is the assumption that allows us to consider indirect comparison as a valid method for comparing two treatments since the studies are sufficiently similar in clinical and methodological characteristics, and therefore in the distribution of effect modifiers.^46,53 Violation of transitivity can lead to inconsistency between direct and indirect estimates for a comparison (loop inconsistency) and/or inconsistency between studies on the same comparison but comparing different sets of interventions (e.g. A vs B and A vs B vs C, called design inconsistency). We will apply both, design-by-treatment interaction model to assess global inconsistency, and local tests on local loops for local inconsistency.^54–56 We will assume common within-network heterogeneity, which is a clinically reasonable assumption since TNS interventions to be included in the network are of the same nature. Heterogeneity will be estimated using the restricted maximum likelihood method.³⁹ We will also conduct meta-regression or subgroup analysis using the same potential effect modifiers as described above to explore evidence of heterogeneity and/or inconsistency.⁵⁴

We will present network diagrams for each outcome and the estimated effects in forest plots. These will be presented with their respective 95% CI, 95% PI, as well as the P-score statistic, corresponding to the percentage of efficacy (or safety) of the underlying intervention, when compared with an optimal intervention without uncertainty. Larger P-score values correspond to better molecules. P-scores across interventions and outcomes will be presented in a rank-heat plot (https://rankheatplot.com/rankheatplot/).⁵⁷ For each pairwise comparison, we will present the direct, indirect, and network estimates.⁵⁸ We will use the netmeta package in R, with R software version 4.3.1 for our statistical analyses.⁵⁹

Certainty in the evidence

We will independently and in pairs assess the confidence in the body of evidence for each outcome using the GRADE approach.⁶⁰ We will grade the confidence in the results of direct comparisons at four levels: high, moderate, low, and very low, based on the ROB 2 tool, which takes into account imprecision,⁶¹ inconsistency (between-studies heterogeneity),⁶² and publication bias.⁶³

To assess the confidence in the estimates, we will scrutinize the confidence in all indirect comparisons derived from the NMA. This evaluation aligns with the GRADE assessment of direct comparisons that constitute the indirect evidence, and consider the judgment of transitivity in this process.^50,64 Confidence in the effect estimate provided by NMA will be considered high when both direct and indirect evidence is available and according, otherwise they will be considered moderate or low or for direct or indirect estimates only respectively.

To grade confidence in indirect estimates, we will focus on assessing the consistency of First-Order Loops (FOLs), those that connect two interventions not directly compared head-to-head through a single common comparator.⁵⁶ For example, if there are trials comparing A vs. C and B vs. C among interventions A, B, and C, we can indirectly estimate the effect of A vs. B through the common comparator C, and thus the indirect estimation AB will be a FOL. We will focus on FOLs with the lowest variances that contribute most to the effect estimates to determine confidence.⁵⁴

Within each FOL, the indirect estimate will be graded based on the lowest confidence level established for its direct components. For example, if confidence in the A vs. C comparison is high and confidence in the B vs. C comparison is moderate, then the confidence in the FOL AB will be moderate. Additionally, if there is suspicion of a violation of the transitivity assumption, the confidence rating of the FOL will be further decreased.

The overall judgment of confidence in an NMA estimate for a paired comparison will be the highest rating between the direct and indirect comparisons contributing to it. However, the confidence rating in the network may be lowered if there is inconsistency between them. In such cases, the GRADE approach recommends carefully evaluating the reason for the inconsistency and which estimate has a lower risk of imprecision based on the available information.⁵⁰

Finally, to facilitate the interpretation of the results and clinical applicability, we will present a summary that categorizes interventions from most to least effective using the minimally contextualized GRADE methodology to draw conclusions from network meta-analysis, which takes into account effect estimates and the certainty of the evidence.⁵⁸