Keywords
Diagnostic Test Accuracy, DTA, R Shiny, ROC Analysis, Sensitivity, Specificity, Predictive Values, Interactive Visualization, Threshold Selection, pROC.
-
Views
-
Downloads
How to cite this article
Quadri SFS, Ahmad M and Lac J. DTAShiny: An Interactive R Shiny Application for Diagnostic Test Accuracy Analysis and Visualization [version 1; peer review: awaiting peer review]. F1000Research 2025, 14:1185 (https://doi.org/10.12688/f1000research.171468.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Software Tool Article
DTAShiny: An Interactive R Shiny Application for Diagnostic Test Accuracy Analysis and Visualization
[version 1; peer review: awaiting peer review]
PUBLISHED 30 Oct 2025
OPEN PEER REVIEW
REVIEWER STATUS
This article is included in the Bioinformatics gateway.
This article is included in the RPackage gateway.
Abstract
Corresponding author:
Joanne Lac
Competing interests:
No competing interests were disclosed.
Grant information:
The author(s) declared that no grants were involved in supporting this work.
Copyright:
© 2025 Quadri SFS et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: Quadri SFS, Ahmad M and Lac J. DTAShiny: An Interactive R Shiny Application for Diagnostic Test Accuracy Analysis and Visualization [version 1; peer review: awaiting peer review]. F1000Research 2025, 14:1185 (https://doi.org/10.12688/f1000research.171468.1)
First published: 30 Oct 2025, 14:1185 (https://doi.org/10.12688/f1000research.171468.1)
Latest published: 30 Oct 2025, 14:1185 (https://doi.org/10.12688/f1000research.171468.1)
Introduction
Diagnostic Test Accuracy (DTA) studies play a pivotal role in evidence based medicine by providing critical information on how well a test can distinguish between individuals with or without a specific condition. Key performance indicators such as sensitivity, specificity, predictive values, and the area under the receiver operating characteristic (ROC) curve (AUC) are used to quantify the test performance. When the test yields continuous results, choosing an appropriate threshold becomes a crucial step, as it significantly influences the calculated metrics and, ultimately, clinical decisions.
While statistical software packages support DTA analysis, few offer interactive tools that allow users to explore the effect of threshold selection dynamically and visualize its impact in real time. To bridge this gap we developed DTAShiny, a web based application built using R and the Shiny framework. DTAShiny enables users to easily upload data, automatically suggests relevant variables, adjust thresholds interactively and generate a wide array of metrics and visualizations.
Methods implementation
DTAShiny is developed using a R programming language (version 4.0+ recommended) and the Shiny web application framework. The user interface is designed with the bsDash package to create a clean, modern dashboard layout that’s both intuitive and responsive.
The core functionalities are supported by several R packages:
• shiny, bs4Dash: For the web application structure and user interface elements.
• tidyverse: Primarily ggplot2 for creating plots (boxplots, histograms, density plots, precision-recall curve, calibration-like plot) and dplyr for data manipulation (e.g., in the calibration-like plot).
• DT (DataTables): For displaying interactive tables of DTA metrics.
• pROC: Used for generating the ROC curve and calculating the AUC.
• haven: For reading Stata (.dta) files.
Data input and variable detection
Users can upload the data in the form of CSV or Stata format. DTAshiny uses simple heuristic rules to automatically suggest which columns represent the reference standard and the index test:
1. The reference standard column: It preferentially selects a column named “status” if it contains binary (0/1) numeric data. Failing that, it searches for other numeric columns containing only 0s and 1s. As a fallback, it selects the first column.
2. The index test column: It preferentially selects a column named “test_value” if present. Otherwise, it chooses the first numeric column that was not selected as the reference standard.
These automatic selections are meant to streamline setup, though users are encouraged to verify their accuracy before proceeding.
Interactive threshold adjustment
If the identified index test variable is numeric, a slider is dynamically generated. This interactive control allows users to explore different cutoff points for classification. The range of the slider is based on minimum and maximum values of the test variable and initially defaults to the median.
DTA metrics calculations
Based on the chosen threshold the DTAShiny constructs a 2x2 confusion matrix (True Positive (TP), False Positive (FP), True Negative (TN), False Negative (FN)). From this it computes the following metrics:
• Sensitivity = TP/ (TP + FN)
• Specificity = TN/ (TN + FP)
• Positive Predictive Value (PPV) = TP/ (TP + FP)
• Negative Predictive Value (NPV) = TN/ (TN + FN)
• Area Under the ROC Curve (AUC) = calculated via pROC::auc()
• F1 Score = 2 × (PPV × Sensitivity) / (PPV + Sensitivity)
• Balanced Accuracy = (Sensitivity + Specificity) /2
• Prevalence = (TP + FN) / (Total observations)
Approximate 95% confidence intervals for sensitivity, specificity, PPV, and NPV are calculated using stats::prop.test.
Visualizations
DTAShiny produces a variety of plots to support interpretation:
• ROC Curve: Plofled using pROC::plot.roc().
• Precision-Recall (PR) Curve: Calculated by evaluating precision and recall over a sequence of thresholds and plofled using ggplot2.
• Distribution Plots: Boxplots, histograms, and density plots of the index test values (overall and stratified by reference status) are generated using ggplot2.
• Calibration-like Plot: Index test values are binned into deciles using dplyr::ntile(). The mean predicted test value within each bin is plofled against the observed proportion of positive cases in that bin. This plot provides a visual, albeit illustrative, sense of calibration.
Operations
DTAShiny can be accessed as a web based application and its source code can be run locally.
Running the application
1. Web version: Available at hflps://786miii.shinyapps.io/786MIIDTA/.
2. Local versions:
• Open-source and hosted on GitHub at hflps://github.com/mahmood789/DTA/tree/main under the MIT License.
• Open app. R in RStudio.
• Run the app using shiny:: runApp().
Interface walkthrough
• Data and Threshold: For file upload (CSV/Stata format). The application automatically identifies the reference and index test variable and creates a threshold slider if the test variable is numeric. The “Run DTA Analysis” initiates the calculations.
• Overview: Displays general information about the uploaded dataset and confirms the variables selected for analysis.
• Standard Metrics: Displays a table of key data metrics (Sensitivity, Specificity, PPV, NPV, AUC) and the confusion matrix. The ROC curve is shown here.
• Advanced Metrix: Presents a table with advanced metrics including F1 Score, Balanced, Accuracy, Prevalence and a table of their confidence intervals. The Precision Recall curve is shown here.
• Plots: Offers visual tools for exploring the data such as: Boxplots of test values by reference group, Histogram of test values, Density plots for test values in each outcome group, A calibration-like plot for assessing the relationship between predicted and observed outcomes.
• Extra Text Output: Includes static text providing general guidance and interpretation notes.
Use cases and illustrative examples
DTA shiny can be utilised in different scenarios:
Initial DTA Exploration: A clinical researcher uploads data from a pilot study of a new biomarker into DTAShiny. They can instantly visualize the test performance, explore how different threshold values affect sensitivity and specificity, and examine key diagnostic metrics all without writing a code.
Understanding Threshold Impact: By interactively moving the threshold slider, users can directly observe how sensitivity and specificity trade off and how predictive values change. This helps researchers to alter diagnostic criteria for their specific clinical contexts.
Visualising Data Characteristics: The boxplots and density plots help in understanding the separation (or overlap) in test values between diseased and non-diseased individuals. The histogram provides an overview of the test value distribution.
Assessing Performance in Imbalanced Datasets: When working with conditions that have low prevalence, the ROC curve can sometimes be misleading. DTAShiny includes the Precision Recall curve which particularly provides more insight in such situations, helping researchers befler interpret model performance.
Educational Tool: Students and Teachers can learn DTAShiny with a simple dataset to learn about the DTA concept, understand how metrics are calculated and see the effect of threshold changes.
Example 1: Biomarker evaluation in screening
A researcher evaluating a new biomarker for early detection of Disease X uploads their pilot study dataset. Using the threshold slider, they explore tradeoffs: 95% sensitivity is achieved at a threshold of 2.6 but specificity drops to 68%. This informs their recommendation to prioritize sensitivity in screening contexts.
Discussion
DTAShiny provides a user friendly and interactive environment for conducting a comprehensive range of DTA analyses. Its main strength lies in its intuitive interface, automatic variable detection, and real time threshold adjustment for continuous tests. By combining standard and advanced performance metrics with rich visual outputs. The inclusion of both ROC and PR curves along with variable distributional plots and an illustrative calibration like plots provides users with a well rounded understanding of test performance.
Strengths
• Interactive Threshold Selection: Allows dynamic exploration of test performance across different cut off.
• Comprehensive Output: Provides standard and advanced DTA metrics, confidence intervals and multiple relevant plots.
• User Friendly: The tool is built around a graphical user interface that minimizes the need for coding along with automatic detection of reference and test variables.
• Supports Common Data format: Accepts both CSV and Stata formats.
Limitations
• Heuristic variable detection: While convenient the automatic detection of reference and test variables, it may not always select the appropriate columns, especially in the datasets with unconventional naming or multiple potential options. Users need to verify these selections before proceeding with the analysis.
• Approximate Confidence Intervals: The CIs for sensitivity, specificity, and predictive values are based on prop.test, which uses a normal approximation. More accurate approaches, like Clopper-Pearson intervals or bootstrap-based methods, could yield slightly different results. The AUC confidence intervals are not displayed explicitly in the current versions.
• Single Test Analysis: The current version focuses on evaluating a single index test against a single reference standard. It does not directly support the comparisons of multiple tests or analyses of paired DTA data.
• No Automated Threshold Optimisation: While the users can explore various thresholds, the app does not automatically calculate or suggest an “optimal” threshold based on criteria like Youden’s J index or proximity to the top-left corner of the ROC space.
• Illustrative Calibration Plot: The Calibration like plot provided in the tool is intended as visual aid rather than a formal calibration test. It does not replace statistical tests such as the Hosmer- Lemeshow test.
Future developments
• Enabling manual selection of reference and test variables, in case auto-detection is inaccurate.
• Implementing more robust methods for calculating confidence intervals, including for AUC.
• Adding functionality to compute and highlight optimal thresholds using formal criteria.
• Extending support to comparisons involving multiple diagnostic tests or test combinations.
• Providing options for handling missing data more explicitly.
• Introducing formal tools for assessing calibration.
Conclusion
DTAShiny is an accessible and interactive R Shiny application designed to empower researchers, clinicians and students performing diagnostic test accuracy analyses without requiring advanced coding skills. By integrating data upload, real-time threshold adjustment, detailed performance metrics, and a wide range of visualizations, DTAShiny streamlines the DTA workflow and enhances the interpretability of diagnostic test results.
Its user-friendly interface, support for common data formats, and comprehensive outputs make it a valuable tool for both research and education. While the current version is focused on single test evaluations, planned enhancements will further expand its capabilities and analytical depth. Overall, DTAShiny contributes meaningfully to the growing toolkit of evidence-based diagnostic methods.
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| F1000Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
Quadri SFS, Ahmad M and Lac J. DTAShiny: An Interactive R Shiny Application for Diagnostic Test Accuracy Analysis and Visualization [version 1; peer review: awaiting peer review]. F1000Research 2025, 14:1185 (https://doi.org/10.12688/f1000research.171468.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
AWAITING PEER REVIEW
AWAITING PEER REVIEW
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Comments on this article Comments (0)