AnVILWorkflow: A runnable workflow package for Cloud-implemented bioinformatics analysis pipelines

Introduction

The NHGRI’s Genomic Data Science Analysis, Visualization, and Informatics Lab-space (AnVIL) consortium was launched in 2018, aiming to democratize genomics data.¹ AnVIL enables easy sharing of genomics data by organizing databases, bioinformatics pipelines for large-scale data processing, and interactive downstream analysis in one Cloud-based platform. AnVIL,² also the name of the platform from the AnVIL project, implements the FAIR data-sharing philosophy and provides a graphical user interface (GUI, supported by Terra³), making it more accessible for researchers without programming backgrounds. However, a GUI tends to be less efficient and slower than a command line interface (CLI), especially for bulk analyses, still requires learning a new platform, and does not support version control and text-based workflows, often included as best practices for reproducible computational research.⁴

Bioconductor’s AnVIL package is an AnVIL API wrapper that provides R-friendly, programming-based functionalities to leverage flexible and scalable cloud-based resources implemented in the AnVIL platform. With the AnVIL package, users can easily access workflows, data, and Cloud-based computing resources managed by AnVIL. However, the AnVIL package is not customized for workflow execution tasks. Instead, AnVIL covers all the resources related to the AnVIL platform, such as interaction with the repository for Docker-based genomic analysis tools and workflows (Dockstore⁵), leveraging cloud resources (Leonardo⁶), and data search and digestion (Gen3⁷). Many AnVIL functions also expose API commands directly, requiring a deep understanding of the underlying AnVIL workspace structures and data models to use for workflow execution. Also, it is a general package without individual support on any workspace and provides no metadata curation. Because most Bioconductor users focus on data analysis, a convenient R-friendly way of accessing and utilizing AnVIL resources is needed. Here, we present the AnVILWorkflow package to meet this need. AnVILWorkflow package is a convenient, fit-for-purpose wrapper around the AnVIL package with the following features optimized for workflow execution:

Users can apply AnVILWorkflow on any workspace they can access, including 347 public workspaces (snapshot on 8.28.23) available to anyone with an AnVIL account. We present three use cases where we ran non-R-based bioinformatics analysis tools using conventional R syntax: Salmon,⁸ bioBakery,⁹ and PathML.¹⁰ Salmon is a widely used RNA sequencing analysis tool for quantifying the expression of transcripts and is based on the command-line interface. Its downstream analysis involves many R/Bioconductor packages, such as DESeq2, edgeR, and limma. bioBakery is a widely used whole metagenomic shotgun (WMS) sequencing data analysis environment, mainly relying on Python. PathML is a general-purpose research toolkit for computational pathology, including many functionalities in digital pathology data analysis, such as strain normalization, nucleus segmentation, and tissue detection. PathML takes raw image files and returns the processed image data in an hdf5 format for further downstream analysis, including machine learning methods.

AnVIL provides comprehensive resources for biomedical data analysis, including data (e.g., genomics), workflows for bulk analysis, and interactive analysis apps (i.e., Galaxy, Jupyter Notebooks, and RStudio) under the workspace. Workflows are often a limiting factor in bioinformatics analysis due to computing demands and the bioinformatics expertise required. Thus, the AnVILWorkflow package makes the workflow-related resources from AnVIL more accessible and easier to use, especially for R users (Figure 1).

Figure 1. Overview of AnVILWorkflow package.

AnVIL’s workflow description language (WDL) specified the runtime environment, which includes proper docker containers for existing analysis tools and computing resources. Cromwell, a scientific workflow execution and management system, runs WDL workflows on the cloud. AnVILWorkflow users can easily run established workflows developed by experts and utilize the cloud resources without configuring or taking maintenance responsibilities.

While AnVIL manages workflow orchestration and workspace metadata and provides default setups that simplify decision-making for users, users still need to manage their data storage and cloud costs. Genomics data, especially their raw and intermediate forms, are very large, so data storage can be costly if the sample size increases. Storage costs incur and can be managed in two ways - storage itself and transfer. For example, using regional storage instead of multi-region, cleaning up intermediate results, and storing infrequently accessed data in low-cost storage (e.g., nearline or coldline storage from Google Cloud) can reduce per-sample costs. Analyzing data stored in one region using Virtual Machine (VM) compute resources in a different region incurs data transfer charges, so centralizing all storage and computing in a single region can be more cost-efficient by not only reducing the storage cost but also avoiding data transfer charges. Currently, the AnVIL workspaces use the us-central1 as a default region, and any artifacts generated from the workflow execution, unless specified, are saved in the same-region bucket linked to the workspace. If users use the default region configured by AnVIL, bringing their data stored in the default region, us-central1, will save the data transfer charge. Additionally, open and controlled access genomic datasets hosted in AnVIL are stored in the us-multi-region, so there are no storage and transfer charges for users using the default workspace configuration. Downloading data to the user’s workstation or laptop is subject to charges, currently $0.08 to $0.12 per GB, depending on the amount of data¹¹ and geography of the transfer, and transfer from the US to another continent is more expensive than within the US transfer.

While browsing existing resources through AnVILWorkflow is free, running workflows charge computing costs. AnVILWorkflow is designed to use existing workflows, which usually predefine computing resources optimized for the types of analyses, simplifying computing-related cost management. You can further reduce the run cost using call caching and preemptive instances. For example, if your workflow runs in fewer than 24 hours since a preemptible VM lasts 24 hours at most, you can save up to 80% by using preemptible VMs.

The cost management for a group of users can be efficiently managed through the AnVIL billing project. One billing account can be shared by adding email addresses under the billing project. The billing project offers details on each workspace, including workspace owner and spent reports, so we can easily identify ‘who’ uses ‘how much’ for ‘what’. In addition to the workspace-level expense reports, users can further enhance cost monitoring by configuring spending reporting.¹² This allows users to closely monitor the expenditure associated with each workflow execution.

Methods

## Setup the account
setCloudEnv(accountEmail = {AnVIL account email},
            billingProjectName = {AnVIL billing project name})

## Clone the workspace of your interest
newName <- {Unique name for your copy of workspace}
cloneWorkspace(workspaceName = newName, templateName = templateName)

## Run workflow
runWorkflow(workspaceName = newName,
            workflowName = {name of the workflow if there is more than one in the workspace of your interest})

## Get workflow outputs
getOutput(workspaceName = newName)

Results

Discussion

The AnVILWorkflow package enables users to conduct complex and computationally intense analyses with minimal bioinformatics expertise through well-established workflows within AnVIL and versatile cloud resources directly from standard laptops using the familiar R syntax. The major advantages AnVILWorkflow provides over the existing approaches include 1) a minimal entry barrier, negating the need for software installations, preparation of properly versioned reference data, or construction and oversight of workflows, 2) leveraging flexible cloud computing resources without the need to learn or handle them directly, 3) user-friendly functions that provide enhanced information, and 4) improved reproducibility and interoperability by seamlessly linking multiple analysis steps, conducted in both R and non-R based tools, within a single R vignette. However, there are still some limitations. For instance, certain customizations of the workflows are limited or require a more profound understanding of the workflows. Despite not being inherently more costly than an in-house server, the pay-per-use structure requires careful planning and management. The absence of an integrated versioning system in AnVIL workspaces requires users to manually monitor new versions. In conclusion, AnVILWorkflow proves most advantages for analyzing a bulk of samples on relatively simple workflows (i.e., single-stage workflow procedure) or for exploratory data analysis for non-technical users, particularly when employing well-established analysis workflows.

Authors’ contributions

SO, LW, and VC conceived the research idea. SO, KG, MR, and MM developed the software. SO and KG performed the benchmarking analyses. SO, LW, and KG wrote the manuscript. LW, SD, MS, and MR reviewed the manuscript.