Artificial Intelligence in the Prediction of Stone-Free Status in Urinary Stone Disease Treated with Extracorporeal Shockwave Lithotripsy: A Systematic Review

Introduction

Urolithiasis ranks as the third most prevalent ailment encountered in urological practice, trailing only urinary tract infections and prostate irregularities in terms of occurrence. This condition stands as one of the most widespread urological maladies worldwide, with estimated prevalence rates spanning from 1% to 13% across various regions.¹ The global incidence, prevalence, and composition of urinary stones exhibit notable disparities and have undergone transformations over recent decades. Specifically, prevalence rates range from 7% to 13% in North America, 5% to 9% in Europe, and 1% to 5% in Asia.² This condition’s incidence is reported to range between 5% and 10%, occurring three times more frequently in men than in women. Individuals aged 30 to 50 face a heightened risk of developing urolithiasis, and it’s noteworthy that some patients experience recurrent stone formation. Most stones are naturally expelled through urination, with the duration of this process contingent on the stone’s size and location. Spontaneous passage occurs in approximately 80% of ureteral stones smaller than 5 mm.³

Various methods are employed to actively remove kidney stones, such as extracorporeal shock wave lithotripsy (ESWL), ureterorenoscopy (URS), retrograde intrarenal surgery (RIRS), percutaneous nephrolithotomy (PNL), and open surgery. While surgical intervention plays a pivotal role in managing urinary stones, it does have certain drawbacks, including a temporary reduction in the patient’s quality of life, prolonged hospitalization, and a high cost burden, thus favoring ESWL as a less invasive alternative.⁴ The concept of ESWL, involving the application of shock waves to disintegrate stones, was initially introduced in Russia during the 1950s and first implemented in humans in 1980. ESWL is considered the primary choice for addressing urinary stones, particularly those smaller than 2 cm, obviating the need for surgical procedures. Success rates for stones smaller than 2 cm hover around 70-80%. However, despite its prevalence, ESWL does exhibit a reported failure rate ranging from 30% to 89% after the initial session.⁵ Effective identification of the most suitable candidates for ESWL can substantially curtail this failure rate, thereby optimizing treatment outcomes and judiciously utilizing limited medical resources.³

To discern the predictive factors affecting ESWL outcomes, numerous studies have concentrated on statistical analyses of patient characteristics employing both bivariate and multivariate approaches. Several factors have emerged as influential in determining ESWL success, including anatomical features of the urinary tract, patient age, stone location, and size.⁶ However, it’s important to acknowledge that these estimates are derived from statistical models based on cohort data, which possess certain drawbacks. These models may overlook significant factors that fall below an arbitrarily chosen numerical threshold, typically 5%. Moreover, as these models employ dichotomized values, clinicians often encounter challenges when attempting to integrate this prognostic information into routine clinical practice.

In recent times, machine learning (ML) algorithms, a subset of artificial intelligence, have found increasing utility in the field of medicine. They are being widely applied to enhance the precision of disease diagnosis and prognosis. Essentially, machine learning comprises a set of techniques that enable computers to acquire knowledge from existing data sets, allowing them to make predictions. What makes machine learning particularly powerful is its capacity to rapidly analyze complex combinations of multiple variables. Among these machine learning algorithms, decision tree analysis (DTA) holds several advantages for medical applications. It is known for its simplicity in understanding and interpretation, as well as its suitability for both qualitative and quantitative data, whether textual or numeric.⁷ Another noteworthy approach is the artificial neural network (ANN), a computational method inspired by biological brains, which approximates how biological neurons tackle problems through interconnected clusters and axons. Each neuron performs a summation function on input values, and the system is self-learning and adaptive, not relying on explicit programming. ANN excels in areas where conventional computer programs struggle to articulate solutions or recognize patterns. Once trained, the network can generate appropriate outputs for given inputs, even when confronted with patterns it has never encountered before. ANN is widely employed as an artificial intelligence technology across various domains.⁸ Random Forests (RF) consist of an ensemble of multiple decision trees working together to establish output classifications. Numerous decision trees are trained using input data, and rather than relying exclusively on the single best-performing tree, a group of them is utilized collectively. Some of the MLs are illustrated in Figure 1. This approach is akin to a committee making decisions collectively, which often results in greater prediction accuracy compared to a single decision tree.^9,10

Figure 1. Illustration of some of the machine learnings.¹¹

The aim of our systematic review is to evaluate the available ML methods capable of predicting the stone-free status in patients with urolithiasis following ESWL.

Methods

Results

Literature search

The initial search across six electronic databases yielded a total of 219 articles, of which 200 were duplicates. After screening the titles and abstracts of the remaining articles, we identified 19 studies that aligned with the criteria set by our systematic review’s PICO framework. However, upon conducting a thorough analysis of the full-text articles, only eleven met our PICO criteria, while the remaining eight studies did not meet the inclusion criteria for this review ( Figure 2).

Figure 2. PRISMA flow chart of including articles.

Risk of bias

This systematic review analyzes the accuracy of machine learning in predicting a condition; therefore, diagnostic articles are included. We use the Non-Proprietary Quality Assessment of Diagnostic Accuracy Studies APPRAISE-AI tool to assess the quality of the diagnostic studies.¹⁸ Out of 11 included studies, 2 are moderate quality, 6 are high quality and 3 are very high. The risk of bias assessment is shown in Supplementary Table 1. Generally, the included studies are of moderate to good quality, only three of which have moderate quality. The risk of bias assessment is shown in Figure 3.

Figure 3. Risk of bias assessment using Cochrane Risk of Bias Assessment.

Only two studies reported the use of additional procedures. Yang et al. performed an additional SWL session every one week if stones remained.⁹ Postoperative stents, catheters, and contrast medium were used by Gomha et al.¹⁵ In three articles, the absence of stones on plain abdominal radiographs determined stone-free status.^12,13,15 Two studies did not state their sampling method.^12,16

Discussion

This review employed broad search criteria and inclusive inclusion criteria to ensure the inclusion of all relevant studies. Based on the Area Under the Curve (AUC) values, the ANN model presented by Michaels et al.¹² achieved the highest AUC. However, it’s noteworthy that this study utilized plain abdominal radiography as the reference standard instead of computerized urography. Interestingly, when plain abdominal radiography is used as the reference standard, the specificity tends to be higher, particularly when the stone is located outside the kidneys. Decision support systems like ANN serve as computer-generated algorithms aiding healthcare professionals in clinical decision-making. These algorithms, in various forms, aim to replicate the learning process of the human brain. They assist healthcare practitioners in making decisions based on specific clinical data from patients. By employing functions within ANN, computers are trained to predict specific parameters efficiently using training sets. Following training, the computer’s performance is evaluated using the data, assessing the extent of its learning in terms of validity and test data. If the computer demonstrates sufficient learning, it can make predictions for users.³

ANN has the potential to greatly enhance the quality of healthcare services, facilitating early disease detection, reducing medical errors, and supporting healthcare authorities in cost-effective patient care. The decision-making process involves selecting one of several alternative outcomes generated by cognitive functions. With an increasing number of alternatives, the complexity of the decision-making process and the potential for errors also rise. At the conclusion of the decision-making process, there is an action or idea. Different formats can be employed to depict how individuals arrive at decisions. It is imperative to scrutinize the interplay between psychological elements, cognitive processes, and the surrounding context when assessing the decision-making process. Individuals are expected to generate specific recommendations through logical filtering and ultimately arrive at the correct decision. Given the value of time in decision-making, providing decision-makers with data as quickly as possible is essential for effective and rapid decision-making. Consequently, many administrative and specialized organizations currently rely on ANN for efficient and swift decision-making.³

Machine learning models exhibit variable diagnostic accuracies, with sensitivity ranging from 35% to 96%, specificity from 63% to 98.4%, and AUC of ROC ranging from 0.49 to 0.96. In this study, it is demonstrated that the predictive accuracy of Random Forest (RF) and Decision Tree Analysis (DTA) in determining stone-free status surpasses that of ANN. It’s worth noting that the high AUC value of ANN was partly attributed to the use of plain abdominal radiography as a reference standard in some studies. Decision tree models present numerous benefits compared to current decision support tools. In contrast to decision tools relying on statistical approaches, artificial intelligence decision models adjust their operations based on the data as they are employed, allowing for the smooth incorporation of new data. Although this adaptability may pose overfitting challenges, setting a minimum number of cases can address this issue. Decision tree models can handle data with both quantitative and qualitative variables, making them versatile. Additionally, results are presented in a straightforward and interpretable manner, often in the form of a tree or a set of rules. Nonetheless, there are certain drawbacks to consider, like attribute importance metrics potentially exhibiting a bias toward variables possessing higher levels of data, including categorical variables with differing value counts.⁷ RF, XGBoost, and LightGBM are three models based on DTA, explaining their comparable accuracy.⁹

Information derived from the analysis of CT images consistently includes crucial predictive factors in the models, primarily mean stone density, stone volume, length, width, and three-dimensional texture analysis. These findings align with current guidelines.¹⁹ Specifically, stone size, volume, length, or width are considered essential predictive factors in eight studies, while stone density is emphasized in three studies, and three-dimensional texture analysis is featured in one study. Incorporating these factors into the model enhances the predictive capability of machine learning for stone-free status.

Several steps are involved in developing AI models. Some of the programs utilized in these studies include Alyuda NeuroIntelligence 2.2 for creating ANN and Quinlan C.50 for producing DTA.^3,7 The algorithm parameters were set to default values to identify factors with the highest predictive accuracy. In the case of DTA, the “minimal cases” parameter defaults to 2 and is used to restrict splits at nodes, leading to the creation of smaller trees as the value increases. By reducing splits at nodes based on this parameter, accuracy gradually decreases, preventing overfitting.7 During the creation of ANN, data were analyzed concerning training, validation, and testing. Both numerical and categorical data were used, and the percentage of data allocated to training, validation, or testing was determined. Subsequently, all data were converted into a numeric format for processing. The ANN structure was then formulated, with the number of neurons determined experimentally, as there are no specific rules in the literature for determining this. The logistic activation function was applied to all inputs and outputs in the ANN model, transforming values into the 0-1 range.³ The RF model, developed by Mannil¹⁰ et al., was created using open-source data mining software (WEKA, version 3.8.0; University of Waikato, Hamilton, New Zealand). A built-in feature selection filter was employed to assess individual features’ value and predictive contribution to SWL (Shock Wave Lithotripsy) success, considering both unique predictive value and correlation between features.¹⁰

This systematic review has several limitations. Many included studies had small sample sizes in both training and testing sets, potentially misrepresenting the AI's true capabilities. The data were predominantly retrospective, introducing potential bias. Methodological issues, including reproducibility, the use of test datasets, diagnostic accuracy reporting, and lack of non-ML statistical comparisons, were also noted. The reproducibility of most studies is limited, with only one study providing model access. This limits the ability to evaluate models with different datasets. There is also significant variability in ESWL outcomes, highlighting the need for consensus to reduce heterogeneity. Furthermore, none of the studies compared their machine learning methods with clinical experts' evaluations, which could provide insights into the effectiveness of AI in clinical settings.

Therefore, future studies should use bigger, prospective, and multicenter data that are externally validated with increased methodological transparency to develop better AI-models. Hybrid models by combining multiple ML approaches should also be explored to improve these existing models. Incorporating standardized reporting of ESWL outcomes predictors will made more progress to increase its clinical applicability.

Conclusion

In conclusion, ML can be used for predicting stone-free status in urinary stone diseases with satisfying accuracy however the accuracy of the prediction rely on many factors such as the ML model, variables taken into account and the data used for training set. The main advantage of ML in the prediction of stone free-status was it allows various factors to be taken into consideration even with the non-linear variables. Random forest method and DTA are superior MLs compared to ANN. Stone size, density, and 3D-texture analysis should be included in the models to ensure accuracy in predicting stone-free status after ESWL. However, access to the models should be made public, and further studies comparing them to the current statistical methods should be conducted.