Keywords
Meta-analysis, Reporting, Interpretation, Large Language Model (LLM), Shiny, R, Automation, Evidence Synthesis, meta, metafor
-
Views
-
Downloads
How to cite this article
khan l, khan M, Rzayev N et al. MetaLLMReporter: An R Shiny App Integrating Meta-Analysis Execution with LLM-Assisted Reporting [version 1; peer review: awaiting peer review]. F1000Research 2025, 14:724 (https://doi.org/10.12688/f1000research.166080.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
Export Citation
Sciwheel
EndNote
Ref. Manager
Bibtex
ProCite
Sente
▬
✚
Software Tool Article
MetaLLMReporter: An R Shiny App Integrating Meta-Analysis Execution with LLM-Assisted Reporting
[version 1; peer review: awaiting peer review]
PUBLISHED 23 Jul 2025
OPEN PEER REVIEW
REVIEWER STATUS
This article is included in the RPackage gateway.
This article is included in the Artificial Intelligence and Machine Learning gateway.
Abstract
Corresponding author:
Joanne Lac
Competing interests:
No competing interests were disclosed.
Grant information:
The author(s) declared that no grants were involved in supporting this work.
Copyright:
© 2025 khan l et al.
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
How to cite: khan l, khan M, Rzayev N et al. MetaLLMReporter: An R Shiny App Integrating Meta-Analysis Execution with LLM-Assisted Reporting [version 1; peer review: awaiting peer review]. F1000Research 2025, 14:724 (https://doi.org/10.12688/f1000research.166080.1)
First published: 23 Jul 2025, 14:724 (https://doi.org/10.12688/f1000research.166080.1)
Latest published: 23 Jul 2025, 14:724 (https://doi.org/10.12688/f1000research.166080.1)
Introduction
Meta-analysis is a valuable method of quantitative data synthesis across studies. For continuous outcome measures, it usually entails calculation and combining either Standardized Mean Differences (SMD) or Mean Differences (MD) (Borenstein et al., 2009). However, merely calculating a pooled average is the starting point. A complete meta-analysis also considers the degree to which large the results of the studies are dissimilar from each other (heterogeneity) (Higgins & Thompson, 2002), whether specific studies can be missing (most likely due to publication bias) (Egger et al., 1997), and to what degree the results are sensitive to including or excluding certain studies (sensitivity analysis). Researchers would also like to see how results of studies differ—this can be explored using subgroup analyses or meta-regression (Thompson & Higgins, 2002; Deeks et al., 2008). In other cases, different approaches like cumulative meta-analysis, Bayesian models, or trim-and-fill adjustments might be helpful. Although excellent R packages like meta (Balduzzi et al., 2019; Schwarzer et al., 2023, 2024), metafor (Viechtbauer, 2010), and dmetar (Harrer et al., 2021) have all these functions, putting it all together into a neat, well-presented report is still hard work. Writing up and summarizing results of heterogeneity tests, bias analyses, sensitivity analyses, and subgroup findings can be tedious—especially if you’re doing language writing for particular journals or for non-technical audiences. To address this challenge, we created MetaLLMReporter.
The R Shiny application (Chang et al., 2024; R Core Team, 2024) automates a wide range of common meta-analyses of continuous outcome data with a single CSV file upload.
The most distinctive feature is interoperability with Google’s Gemini large language model (LLM). After completing the analyses, the app then gathers all the results and sends them to the LLM to generate draft reports in various formats—plain language summaries, for example. This allows researchers to produce written descriptions of their results quickly without wasting time and improving readability.
Methods
Implementation
The app is built using R (version 4.4.0) and the Shiny web framework (version 1.8.1.1). It uses the bs4Dash package (version 2.3.2; Granjon, 2023) to create the dashboard interface.
Core meta-analysis engine
A full suite of analyses is run using functions from established R packages:
• Main Pooling: The function meta::metacont is used to pool MD or SMD values, depending on user selection. The user also selects the effect measure, pooling method, heterogeneity estimator, and whether to use a fixed or random effects model.
• Heterogeneity: Key heterogeneity statistics (Q, I2, and τ (Borenstein et al., 2009)) are taken from the metacont object.
• Sensitivity: A leave-one-out analysis is done using meta::metainf, and outliers are detected using dmetar::find.outliers.
• Publication Bias: Egger’s regression test is done with meta::metabias, and trim-and-fill analysis with meta::trimfill. P-curve analysis uses dmetar::pcurve.
• Meta-Regression: If columns Reg, Reg2, or Reg3 are available, meta-regression is done using metafor::rma.
• Subgroup Analysis: If a subgroup column is provided, subgroup analysis is performed using meta::metacont.
• Cumulative Analysis: This is handled using meta::metacum.
• Bayesian Analysis: The Bayesian meta-analysis uses bayesmeta::bayesmeta (Röver, 2020), with effect sizes first calculated using metafor::escalc (Lüdecke, 2018).
Data input
Users upload a CSV file via shiny::fileInput. The data must contain columns for meanintervention, sdintervention, totalintervention, meancontrol, sdcontrol, totalcontrol, and author. Optional columns include Reg, Reg2, Reg3, and subgroup. If needed, a default value can be set for totalintervention (Wickham et al., 2023, 2024).
LLM integration
API call
The helper function generate_content is used to send a request via httr::POST (Ooms, 2024), using jsonlite::toJSON (Ooms, 2014) to format the request and fromJSON to process the response. The Gemini API (specifically the gemini-2.0-flash model) is called using a randomly selected API key from a hardcoded list. The user need to insert their own API key if using from github.
Prompting
After the statistical analyses are completed reactively, their textual outputs are collected using capture.output and combined into a single string (allText()). This string is used as the main input for the LLM. Prompts include specific instructions to rewrite the content in different styles (Cochrane, NEJM, Lancet, or Plain Language), along with guidelines for structure and minimum length.
Output processing
The LLM’s raw response is cleaned of basic markdown formatting using the clean_markdown function (based on gsub), and the cleaned output is rendered using shiny::htmlOutput so that formatting like line breaks is preserved (Xie et al., 2024).
User interface
The dashboard interface includes two main tabs:
• The “Inputs” tab handles data upload and lets users choose key meta-analysis settings.
• The “Meta-Analysis Text” tab displays the outputs for each analysis type (Heterogeneity, Leave-One-Out, Publication Bias, etc.), along with four tabs that show the LLM-generated text in Cochrane, NEJM, Lancet, and Plain Language formats.
Operation
You can run MetaLLM Reporter locally with R (version 4.0.0 or higher) and the required packages, or use a hosted version in your browser. To access LLM features, you’ll need an internet connection and a valid Google Gemini API key (user to provide).
Data upload & settings
In the “Inputs” tab, upload a CSV with mean, SD, and sample size for both groups, plus the author name. Optional columns (Reg, Reg2, Reg3, subgroup) enable meta-regression and subgroup analysis. Select your effect measure (MD/SMD) and analysis options. A sample CSV is available for reference.
Use cases
MetaLLM Reporter is designed to assist researchers who have completed a meta-analysis, or possess the requisite data, in consolidating diverse statistical outputs into comprehensive written summaries or reports.
Scenario 1: Drafting a results section: A researcher uploads their meta-analysis dataset. After confirming the analyses run correctly by checking the individual output tabs (e.g, Heterogeneity, Publication Bias), they navigate to the “LLM Cochrane” or “LLM NEJM” tab. They copy the generated text as a starting point for drafting the results section of their systematic review manuscript, carefully reviewing and editing the LLM output for accuracy, completeness, and appropriate nuance.
Scenario 2: Plain language summary generation: For patient education materials or conference abstracts, their searcher validates the “LLM Plain” tab output. This concise summary of the meta-analysis findings is meticulously reviewed for accuracy against the statistical results.
These scenarios demonstrate the tool’s capacity to automate the generation of initial text drafts from complex statistical results, potentially expediting the reporting process.
Analysis
Though a vast number of tools exist for doing meta-analysis, briefing results from heterogeneity tests, bias calculations, sensitivity analyses, and subgroup/meta-regression analyses into intelligible written description relics a massive undertaking.
MetaLLMReporter referrers to this problem by performing not just a plethora of average meta-analysis actions using already existing R packages (meta, metafor, dmetar, etc.) but also by mixing a large language model (LLM) to instinctively produce reports in different styles.
The main revolution is the usage of Google’s Gemini LLM to produce statistical yields into organized texts that are catered for a diverse audience, going from recognized Cochrane-style reports to simple language abstracts. This intends to drastically decrease the writing effort that is expected by researchers, providing them with drafts, including results from various analytical components. The program executes a wide range of appropriate analyses robotically on data upload, making sure the components are readily accessible for the LLM prompt.
Nonetheless, the dependence on LLM-generated texts needs serious deliberation. Whilst possibly saving time, the output must be cautiously reviewed as a draft needing thorough analysis and proof reading by experts. LLMs may mistake the context, misunderstand statistical distinctions, or quite possibly “hallucinate” information. The superiority of the LLM is seriously reliant on the cohesive and extensiveness of the prompt given and with the abilities of the LLM used. The present implementation also necessitates implanting API keys, leading to potential security risks thus making it inappropriate for large scale sharing without changes made. Moreover, the app centres on text production and lacks the interactive plotting aspects found in other meta-analysis tools.
Changes down the line could potentially give users the ability to choose which analysis to incorporate in the LLM prompt, fine tuning the prompts for more precision and style adherence, including visuals together with texts, accompanying further data types, presenting options for diverse LLMs, and applying additional safe API key management.
Ethics approval and consent to participate
This study did not involve human participants, human data, or human tissue. Therefore, ethical approval and consent to participate were not required.
Software availability
Source code available from: https://github.com/mahmood789/LLMSMD
Archived software available from: https://doi.org/10.5281/zenodo.15790137
License: Apache 2.0 (OSI-approved open license)
This software is freely available and can be used without restriction. Use of the LLM features (e.g., Gemini API) requires a valid API key provided by the user. The demonstration dataset and all analysis functionality are included in the GitHub repository for full replication of results.
Download
Export To
metrics
| Views | Downloads | |
|---|---|---|
| F1000Research | - | - |
|
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Citations
CITE
how to cite this article
khan l, khan M, Rzayev N et al. MetaLLMReporter: An R Shiny App Integrating Meta-Analysis Execution with LLM-Assisted Reporting [version 1; peer review: awaiting peer review]. F1000Research 2025, 14:724 (https://doi.org/10.12688/f1000research.166080.1)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.
Open Peer Review
Current Reviewer Status:
AWAITING PEER REVIEW
AWAITING PEER REVIEW
Key to Reviewer Statuses
VIEW
HIDE
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations
A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Comments on this article Comments (0)