This review followed PRISMA guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and The Cochrane Handbook for Systematic Reviews and Meta-Analyses. ^{27, 28} This review was registered on the PROSPERO website (Registration No. CRD420251139903).

Systematic searches of PubMed, Scopus, ScienceDirect, and ProQuest were conducted for each database through August 2025. Medical Subject Headings (MeSH) and a combination of free-text words were used in the search, without any language limitation. The detailed search strategy is available as Extended data (Appendix 1: Search strategy).

Eligible studies were restricted to RCTs examining digital CDSS applied for clinical decision-making in CKD, including EHR/CPOE-interfaced solutions and web-based or mobile (standalone) electronic decision supports. The interventions might be assisted by AI/ML or rule-based. Trials had to have at least one process domain concerning medication safety. The primary outcome was appropriate dosing according to renal function, and secondary outcomes included CKD identification/documentation, death rate, renal function (as measured by serum creatinine/eGFR), BP/CV events and usability of the CDSS as well as implementation outcomes such as adherence to or compliance with recommendations made by the CDSS when reported. Non-randomized studies, non-electronic databases, and those not specifically for CKD were excluded. Details are available in Tables 1 and 2.

Data were imported into Covidence and deduplicated, where data were also screened (titles/abstracts) with full texts reviewed for eligibility. A standard Excel form was used to record study features (title/authors/year, setting/country, RCT design/participants), CDSS features (functionality/classifications/intervention(s)), comparator(s), outcomes (primary/secondary), data types, and arm-specific sample sizes/follow-up. Screening and data extraction were performed by two independent reviewers, with a third reviewer to resolve any disagreements.

Risk of bias across studies was assessed with Cochrane’s RoB 2 for the following five domains: randomization process, deviations from intended interventions, missing outcome data, measurement of outcome, and selection of reported results. Risk of bias (RoB) judgements were performed using the RoB 2 guidance: if all domains were at low risk, then the overall RoB was classified as low; if one or more domains were judged to be high, the overall judgement was high; otherwise, some concerns. Methodological quality was assessed by two reviewers working independently of one another, and disagreements were resolved by discussion or a third reviewer.

We did not formally assess publication bias (e.g., funnel plot–based methods) because the number of studies contributing to each synthesis was limited and the included trials were highly heterogeneous in interventions and outcomes.

Meta-analyses were performed using RevMan 5.2 (accessed September 2025) with fixed- and random-effects models. Dichotomous results were aggregated as risk ratios using the Mantel–Haenszel test; continuous ones were calculated as MD (or SMD). We used fixed effects for low heterogeneity (I ² ≤ 50% and conceptually homogeneous outcomes) and random effects models for high heterogeneity (I ² > 50%) or important design/setting variation. For uncertainty, 95% CIs were provided; for heterogeneity, I ² and Chi ². A 0.5 continuity correction was applied to zero cells when necessary, and multi-arm studies were combined as per the Cochrane Handbook to prevent double-counting. Due to the substantial heterogeneity we found in our main effects, we also presented pooled effect estimates with prediction intervals (PI) for a better appreciation of expected results across various clinical settings. As the pooled estimates themselves provide limited refinement of information, due to how implementation is nuanced and varied at each site, our meta-analysis was also given support with a structured Best Evidence Synthesis (BES) and trial-level mapping by delivery mode and workflow stage, incorporating TIDieR elements; trial network mapping was performed using Cytoscape (version 3.10.4).

We did not apply GRADE because outcomes and effect measures were highly heterogeneous and many trials did not report estimable effect sizes suitable for consistent certainty rating across outcomes.

Based on the developed search strategy, 1,288 articles were identified through systematic searching. Of these, after removing 173 duplicates, 396 articles were screened by title and abstract. Then, for reasons not meeting the inclusion criteria, 284 were excluded. We sought retrieval of 112 reports, of which 75 were not retrieved, leaving 37 reports assessed for eligibility. In total, 20 trials met our inclusion criteria, and 6 of them underwent quantitative analysis. Figure 1 depicts the process of literature retrieval.

The main characteristics of the included studies are summarized in Table 3. The CDSS interventions we examined were heterogeneous in terms of their technical platform (eg., combined EHR/CPOE systems to standalone applications for mobile platforms) and core functions (eg., renal dosing support or CKD documentation). To consider this diversity as the first major contributory source of heterogeneity, we sought to control for it upfront using an organized Best Evidence Synthesis (BES).

To provide a view beyond common pooled estimates, we used a layered model for evidence synthesis. This comprised an Evidence Map ( Figure 4) to provide an overview of coverage across outcome domains, and a Best-Evidence Table ( Table 7) identifying robust patterns and areas of poor reporting within each configuration. Finally, we illustrated the ‘architecture’ of this evidence with a Network Map ( Figure 5) that outlines how details from particular trials and implementation configurations convert into outcome domains.

For the six studies included in the meta-analysis, the risk-of-bias assessment is shown in Figure 2. For the first outcome, 4 RCTs were generally at high risk or unclear in several domains (e.g. deviations from intended intervention, missing outcome data, selection of the reported result); underestimation of the outcome was largely low risk ^{29– 32} ; this corresponds with very high heterogeneity observed (I ² = 97%; pooled RR 1.76, 95% CI 1.13–2.74). For the second outcome, Peralta (2020) was low risk across all domains, whereas Sperl-Hillen (2023) identified several domains as high risk. ^{33, 34} Effect consistency in EHR-documented CKD banner was high (I ² = 0%) with precise estimates (fixed-effect: RR 1.20 [1.08–1.33]; random-effects: RR 1.19 [1.07–1.32]). Risk-of-bias diagrams are shown below ( Figure 2).

3.4.1. Appropriate renal dosing

CDSS significantly improved appropriate renal dosing compared with standard care, with a relative risk (RR) of 1.76 (95% CI 1.13–2.74; Z = 2.51; p = .01). But there was considerable heterogeneity among the studies included, and I ² = 97%, τ ² = 0.19, χ ² = 72.66, df = 3, p < 0.00001. Therefore, a random-effects model was used to pool results across four RCTs. ^{29– 32}

The effect of these interventions was highly varied; study-specific effects ranged from RR 1.20 to 3.00. This wide range illustrates how success is highly influenced by the clinical context and implementation approach. In absolute numbers, the CDSS arms reached appropriate dosage in 2,926 out of 5,565 medication instances-significantly more frequent than the same figure in control groups (2,230/6,588) ( Figure 3).

3.4.2. Prescribing and medication safety

Most studies show that an advantage works for rule-driven CDSS. In seven studies assessing for the dosage/prescription of medication, five were in favor of CDSS, including reductions in overdose (19.2% vs 34.5%, OR 0.45; p < 0.001), ³⁵ reductions in excessive dosage (43% vs 74%, p = 0.001), ³¹ fewer medication errors (33% vs 49%, p < 0.001), ³² various measures for proper dosage across many subcomponents (RR doses 0.95; Frequency 2.4; Avoidance 2.6; Information 1.8), ³⁰ and higher rates for suitable renal dosage (17% vs 5.7%, OR 1.89; 95% CI 1.45–2.47; p < 0.001). ²⁹ One study found no significant difference in prescription failure (1.26% vs 0.5%, p = 0.11), ³⁶ and another was neutral regarding workflow/guideline outcomes. ³⁷ Beyond dosage/prescription, one hemodialysis study reported reduced severe hypotension (8.3% vs 13.8%; p = 0.01), ³⁸ and one study of nutritional/biochemical changes showed improvements in parameters (albumin, prealbumin, hemoglobin, BUN), anthropometrics, quality of life, and satisfaction. ³⁹ One risk prediction study did not apply clinical implementation, ⁴⁰ while the other provided no outcome data or direction of effect. Overall, evidence in the domain of medication safety and appropriateness supports CDSS; however, effect sizes and outcome types differ across studies.

3.4.3. Best Evidence Synthesis (BES) by implementation configuration

Magnitude of effect was diverse, with study-specific RRs ranging from 1.20 to 3.00. While the summary of raw values points towards a benefit in the CDSS arms (2,926 doses correct out of 5,565 as compared to 2,230/6,588 in controls), heterogeneity is marked (I ² = 97%) and any one average effect is virtually meaningless. This initiates our attention for a Best Evidence Synthesis, to investigate why certain settings work better than others.

Our mapping of trials by delivery method and workflow timing reveals a lopsided evidence base. While there is no shortage of data on how interruptive alerts affect prescribing errors, the evidence regarding long-term clinical significance and implementation hurdles remains thin and varies wildly between settings (see Figure 4). This gap between process-level success and clinical impact is critical. We’ve summarized the specific signals and reporting deficiencies for each configuration in Table 7. Finally, Figure 5 gives a network-level perspective on the ‘architecture’ of the evidence base; we can see which trials support each configuration, and in what area of the evidence base fragility remains. A more detailed trial-by-trial mapping can be found in the Extended data (Appendix 2).

3.5.1. EHR-documented CKD recognition

CDSS performed significantly better than standard care in identifying documented CKD in the EHRs (relative risk [RR] = 1.19, 95% confidence interval [CI] = 1.07 to 1.32; Z = 3.30, p = 0.001). There was no heterogeneity (I ² = 0%; τ ² = 0.00; χ ² = 0.53; df = 1; P = 0.47). Pooled data from two RCTs were analyzed using a random-effects model based on weighted inverse variance with restricted maximum likelihood (REML) estimation. The study-specific estimates were Peralta (2020) ³⁴ (RR = 1.11 [0.90–1.38]; weight, 24.5% [86/165 vs 88/188]) and Sperl-Hillen (2023) ⁴¹ (RR = 1.22 [1.08–1.38]; weight, 75.5% [417/1568 vs 389/1783]). A total of 503 of 1733 events occurred in the CDSS arms vs. 477 of 1971 in the control arms. A summary of aggregated effects under fixed- and random-effects models is presented in Table 4.

3.5.2. Mortality events

Two studies provided mortality data. There was no difference in mortality between AI-CDSS and conventional care in our randomized hemodialysis trial (one case in each group). Other significant adverse events, such as hospital admission and blood transfusion, also showed similar results (7 versus 9 and 5 versus 5). ³⁶ Second, the other study did not assess the impact of the CDSS intervention in clinical practice but rather aimed to develop and validate a machine-learning-based risk algorithm. Mortality (renal and cardiovascular) was a non–treatment-related available future risk predictor, but there was no clinical practice evaluation of the CDSS, ⁴⁰ Consequently, it can be concluded that we were unable to assess intervention effect on mortality, despite its inclusion in that trial among the predicted composite outcomes assessed from treatment or control status; however, as alluded to earlier this means that study had to be excluded from the meta-analysis for subject outcome specifications on Mortality in Table 5.

3.5.3. Renal function

The evidence is very suggestive of a beneficial effect of the intervention groups on general renal function and certain specific physiological outcomes. However, the findings show varied results across different types of outcomes. The benefits of the CDSS intervention were described in two studies. One trial that used a rule-based CDSS reported the estimated Glomerular Filtration Rate slope to decline less steeply when facilitation was added (−0.01 with CDSS plus facilitation vs − 0.95 with CDSS alone; p < 0.001; favors intervention), ⁴² Another AI-CDSS study also showed an increase in renal cortical blood flow (p < 0.05; between groups). ³⁹ Three other studies were of neutral or no difference around hemoglobin control (within target with AI-CDSS versus physician management), ³⁶ CKD quality metrics, ³³ and laboratory monitoring. ⁴³ One study was for risk prediction model development, not clinical implementation, therefore presenting no comparative effect data; and the other did not report the data or direction for the outcome. ⁴⁰

3.5.4. Blood Pressure (BP) control and Cardiovascular (CV) outcomes

The evidence for the potential utility of Clinical Decision Support Systems (CDSS) across a range of CV and BP-related outcomes is cumulative. At the same time, results vary across BP measures in the included studies—mixed directionality of effect. Five studies showed mixed effects: three favoured the CDSS and two were neutral. Regarding blood pressure-related outcomes (change or control; three studies), one showed a decrease in systolic blood pressure (SBP) that was significantly greater among the CDSS arm as compared to the standard care group (-14.6 mmHg for CDSS vs -11.7 mmHg for control; p = 0.005), ⁴¹ whereas two others did not find differences in terms of either blood pressure control ⁴² or change. ⁴⁴ Furthermore, a single study reported improved quality of life (favoring AI-CDSS), ³⁹ and another study reported a decreased rate of severe intradialytic hypotension (also favoring the AI-CDSS). ³⁸ Quantitative effect sizes and p-values were reported exclusively in one study; the other four reported only the direction of effect without providing numerical values.

3.5.5. Usability and provider acceptance

Clinicians’ involvement in CDSS was highly heterogeneous in this context. Provider-level adherence in five studies with quantitative data ranged from 17% to 74%. ^{29, 31, 34, 45} Only two studies included qualitative terms to define engagement (low use and high compliance) ^{43, 46} while four studies reported no adherence outcomes. ^{32, 33, 36, 37} User satisfaction was quantitatively evaluated in two studies and showed low or no burden, with high satisfaction rates of 74% and 82% among users, ^{37, 44} respectively. Barriers for (non-)implementation and use were reported in eight studies, among others, alert fatigue, lack of training, time pressures, unfamiliarity with CDSs or resistance to change, technical problems like application crashes, disabling printing, economic reasons, disturbances relating to COVID-19, and overall burden of users on time. ^{29, 31, 33, 36, 37, 43, 45, 46} Regarding the facilitators, they were cited in two studies that reported integration of their results and a reduction in false positives. ^{32, 34} Sustainability was reported in three papers: two reported plans to continue or increase use, and one found sustained error reduction. ^{31, 32, 46} In general, clinician acceptance of CDSS was moderate and varied considerably; there is potential to further support implementation by integrating systems to improve CDSS adoption.

3.5.6. Guideline adherence and clinical process measures

For guideline adherence and process measures, two studies favored AI-CDSS implementation, two were neutral, and one reported no effect direction. AI-CDSS was linked to increased ACEi/ARB utilization and nephrology referral, and decreased medication errors. ^{32, 47} In contrast, adherence to ESA was not inferior to usual care or low, which probably reflects human/systemic barriers. ^{36, 45} Heterogeneity of outcomes prevented pooling; overall, benefits seem to be aimed at prescribing/referral and medication security, with adherence possibly requiring more than decision support ( Table 6).

The most consistent benefit to patient safety is better renal-dose appropriateness and its prescribing. Studies that included real-time alerts or order-entry checks resulted in higher doses based on kidney function, decreased use of contraindicated drugs, and reduced excessive dosing of renally eliminated drugs. ^{29– 32, 35, 38, 39} As these errors represent a standard route to drug-induced AKI and other serious events, such process gains are clinically relevant even when not all of the adjudicated ADEs have been captured. The substantial heterogeneity in early dosing trials, however, suggests that impact depends heavily on local thresholds, targeted drug classes, degree of EHR integration, and provider response to prompts. ^{29– 32, 35} Consistent with this, our configuration-based Evidence Map ( Figure 4) indicates that the densest evidence for process improvement clusters in interruptive, order-entry implementations, whereas clinical outcomes are reported far less consistently across configurations.

By contrast, effects on renal function, cardiovascular events, and survival were inconsistent. A small number combined a CDSS with other interventions, such as practice facilitation or intensive management, and found positive eGFR trajectories and improved renal cortical blood flow, or less severe intradialytic hypotension, but negative hemoglobin control, CKD quality measures, and survival. ^{33, 36, 38– 40, 42– 44, 48} This discrepancy between relatively obvious benefits in proximal safety process measures and neutral downstream outcomes is not surprising given that trials were powered on process endpoints; the follow up periods were short; the CKD populations studied were heterogeneous with co-morbidities and competing risks; and those running concurrently other quality initiatives (even significant reductions in dosing error rates might equate to relatively minor absolute alterations in hard outcomes).

Figure 5 further confirms the fragility of the evidence base. By visualizing the network at the trial level, we observe that clinical-stage testing is restricted to a substantially smaller number of configurations than the raw trial counts might imply. This raises a Very Big Risk: it is the nature of our beast that we aggregate patient-level data from a literature dominated by process-level data, and we do so with extreme caution.

CDSS also slightly increased EHR-documented CKD recognition in trials that focused on problem-list documentation and risk stratification; being labeled appropriately is a condition precedent to activating renal-dosing rules, monitoring protocols, or nephrology referral prompts. ^{33, 34, 42, 44, 48} The majority of studies considered transparent rule systems that capture dosing thresholds or contraindication-related information, whereas AI-enabled tools focused on higher-level predictive tasks, such as optimizing erythropoiesis-stimulating agent dosing or estimating cardio-renal risk, rather than broader formulary-wide dose checking. ^{29– 32, 35, 36, 38– 40, 42, 48} There is randomised trial evidence that AI systems are no less effective than standard practice at focusing on process-level and physiological surrogates. To date, we have not seen any full publication of an RCT showing a clear benefit over a well-designed rule-based system in patient-centered safety. This is a crucial distinction. With this in mind, Table 7 reflects “evidence signals” that are configuration-specific, as well as ongoing reporting gaps that continue to enable us to ascertain whether a system’s success is due to its algorithm or to better workflow integration.

Implementation and usability results provide insight into why the safety potential of CDSS is incompletely realized. Adherence of providers to the CDSS’s advice varied from very low to moderate. Two main and disadvantageous themes (alert fatigue, time pressure) as well as seven other recurring patterns in the qualitative findings (lack of training, weak technical integration, limited use or coordination with pharmacists, limited involvement in nephrologist consultations) were identified. ^{29, 31– 33, 36, 37, 43– 46} High rates of non-specific alarms can desensitize clinicians and add to alert-fatigue burden (e.g., hazard related to automation when critical warnings are missed), unclear attribution for responding to high-risk activations, and weak embedding within multidisciplinary care can further dampen effects on patient clinical outcomes. ^{32, 36, 45, 47} This study does not characterise CDSS as software. Still, as a socio-technical intervention, it draws attention to the fact that safety is as much about governance and workflow as it is about algorithmic accuracy. This transition in perspective is also why we deviated from treating transfer modes and workflow kinetics as frozen “background” variables and chose to treat them as dynamic, living entities. To put these findings into context, Extended data (Appendix 2: Trial-level mapping table) of the current study offers an in-depth examination of the trials included, while ‘seeing beyond the tell tale’. Deconstructing how this evidence was collected, we seek to distil lessons for what our ‘current’ evidence base can actually support–an exercise we see as important in the face of recent technological developments that could potentially render such data obsolete.

To health systems and quality-improvement leaders, none of this proves that renal-dose CDSS raises survival. Current evidence suggests treating the thing as safety-critical infrastructure whose primary value in tertiary analyses may be preventing avoidable prescribing errors such as overdose of renally cleared agents, failure to adjust dosing down with falling eGFR, lapse in initiation of ACEi/ARB therapy for most indications other than AKI with HFrEF, or lag in being referred to nephrology. ^{29– 32, 35, 38, 39, 45, 47} Deployment should accompany medication safety initiatives in programs, including pharmacist-led review of high-risk prescriptions, standardized CKD order sets, and promoting drug-kidney education. A few simple metrics, the proportion of renally inappropriate orders, patterns of overrides for high-risk alerts, and serious medication-related events can recalibrate thresholds, eliminate low-value rules, and pinpoint services calling for support. In contrast, alert design should focus on high-risk scenarios with straightforward recommendations to incorporate into the workflow of ordering. ^{29, 31– 33, 36, 37, 39, 40, 42– 46, 48} Our configuration-based synthesis further suggests that implementation decisions (e.g., interruptive alerts at order entry vs non-interruptive dashboards) should be considered part of the intervention itself, because they condition adoption and the plausibility of achieving clinically meaningful downstream impact.

Three guiding considerations arise from the existing evidence. ^{29– 32, 35, 36, 38, 40, 42, 48} Pragmatic (sufficiently powered, head-to-head RCT) comparisons between AI-CDSS and mature rule-based systems are required to establish whether advanced prediction and personalization provide incremental safety or cost-effectiveness gains beyond those achieved with effectively implemented rules. Second, a CKD-CDSS core outcome set, which includes measurement of renal-dose accuracy: EHR-documented CKD recognition; nephrology referral; blood pressure control; and important patient safety outcomes such as acute kidney injury (AKI), serious ADRs, or MHRs, improves comparability of studies. Third, new trials should adhere to such frameworks as CONSORT-AI for designing more complex interventions that establish implementation parameters as primary outcomes driven by mediation analyses linking prescription-process gains to clinical endpoints, with consideration of transparency, bias (eg, endogenously impacting trained vs nontrained sites), data drift crossover, and Equity in AI-CDSS models. To address the recurring gaps in the evidence observed ( Table 7, Extended data in Appendix 2: Trial-level mapping table), future trials may need to include better reporting of outcomes. This encompasses detailed trigger logic, the alerting delivery mechanism, and levels of severity, as well as a mindset toward the reporting of override data. And indeed, the slogan that ownership lineage and governance for model updates are a substitute for what is wanted has proven to be very true. These are the factors that determine whether a study’s data can contribute to interpretation or be included in subsequent meta-analyses.

This systematic review was reported according to the PRISMA guidelines, was prospectively registered in PROSPERO, and included only randomized controlled trials, thereby improving internal validity compared with mixed-design reviews. When outcome-specific RoB 2 evaluations were applied and random-effects models with fixed-effect sensitivity analyses fitted, there was evidence that process outcomes differed from clinical endpoints and a physical apparent pattern in time; i.e., earlier dosing studies yielded larger, although more heterogeneous effects, while more recent EHR-integrated trials gave rise to smaller but less diverse gains in CKD awareness and associated processes. Another strength is the structured nature of the synthesis and the visualization of the architecture of the evidence at the trial level ( Figure 4 and Figure 5, Table 7). More than just creating identifiable pooled effects, these methods provide a nuanced tool for navigating research in which heterogeneity is so vast. They guard against the interpretive myopia that would allow our calculations to be reported but not properly taken into account in their specific context.

It is important limitations must also be recognized. There was significant heterogeneity across many of the most important outcomes, driven by differences in outcome definitions and trial design; only a proportion of eligible trials contributed to each meta-analysis. In particular settings, there were too few studies with effect estimates to allow informative subgroup analyses or exploration of publication bias. Key domains were at risk of “some concern” or “high” bias in several studies, and the majority of studies took place in high-income countries with well-established EHR systems. Few trials explicitly quantified adjudicated adverse drug events and other patient-centred safety measures, so estimates of harm reductions continue to be based largely on the premise that improvements in prescribing practice translate into less harm. ^{29– 32, 35, 38} We acknowledge, however, that the most consistent signal is around improved renal dosing and related prescribing practices. Finally, because trial reporting of configuration and implementation details was frequently incomplete, the Evidence Map and network visualization necessarily reflect what was reported; Table S2 is therefore provided to transparently document reporting presence/absence and to guide more reproducible future evaluations.