Workflow languages are widely used when production systems analyze biomedical data since they provide consistent processing of batches of data, reproducible processing of a particular dataset, restart and reentry, data provenance, and other desirable properties, such as the ability to easily share bioinformatics pipelines with others. ¹

For this reason, a wide variety of workflow languages and workflow managers are used in bioinformatics. For example, the most complete list of computational data analysis workflow systems ² contains 350+ entries.

The popularity of particular workflow languages and workflow managers varies by subfield, data type, computing environment, and over time. Table 1 summarizes the number of workflows by used languages in the main workflow registries related with bioinformatics: Dockstore ³ and WorkflowHub. ⁴

Here we briefly describe the top 4 workflow languages listed in Table 1 and corresponding workflow systems.

WDL - Workflow Description Language ⁵ was originally developed for the Broad Institute’s genomic analysis pipelines. Now it is an open standard for describing data processing workflows with a human-readable and writeable syntax. WDL makes it straightforward to define analysis tasks, connect them together in workflows, and parallelize their execution. The language enables common patterns, such as scatter-gather and conditional execution, to be expressed simply. ⁶

To execute WDL scripts there are two main workflow engines: -

Cromwell ⁷ - is a production-grade workflow execution engine developed by the Broad Institute. It is written in Scala and is designed for high-scale execution of WDL workflows, particularly in cloud environments like Google Cloud Platform (GCP) or on local clusters. Many users run their WDL workflows in Terra, ⁸ a managed cloud bioinformatics platform with built-in WDL support provided by Cromwell.

CWL - Common Workflow Language ^{10, 11} is another open, community-led standard designed for describing analysis workflows and command-line tools in a portable and scalable way. CWL workflows use YAML syntax to define inputs, outputs, and execution steps, allowing them to be run consistently across different computing environments, including workstations, clusters and clouds. cwltool ¹² is the reference implementation of the CWL standards. Other implementations are listed at the dedicated website. ¹³

Nextflow ^{14, 15} is an open-source workflow management system and a domain-specific language (DSL) designed for writing and executing scalable and reproducible scientific workflows. It is based on a reactive dataflow programming model, where tasks are triggered automatically when input data becomes available or changes, allowing natural parallelism without the user having to manage low-level details of task scheduling or resource locking. The DSL, based on Apache Groovy, lets users define workflows that integrate scripts in multiple languages (bash, Python, R, Ruby, etc.) and can reuse existing software tools without wrappers.

nf-core ^{16, 17} is a community-driven project that provides a curated set of standardized, best-practice analysis pipelines built using Nextflow. Significant effort has gone into developing tools, templates, and guidelines that enable domain experts to contribute to the community. The result is a set of high-quality pipelines that are portable, reproducible, fully documented, and cloud-ready.

Seqera ¹⁸ is the company behind Nextflow and has supported the nf-core community since its inception. It has also developed the commercial workflow system Sequera platform and a powerful AI tool specializing in bioinformatics and nextflow language in particular - Sequera AI. ¹⁹

Galaxy ^{20, 21} is an open-source, web-based bioinformatics workflow system implemented using the Python programming language. It is designed for accessible, reproducible, and transparent computational research. Its key feature is graphical workflow creation. Galaxy makes thousands of third-party open-source analysis packages easy to use, and interoperable without any user supplied code. For any new analysis package to become a tool, a developer prepares a Galaxy wrapper once, and uploads it to the sharable Galaxy global tool ‘appstore’ called the Galaxy Toolshed. ²² However it uses proprietary JSON format to describe workflow components.

It is desirable to have reliable tools that translate a workflow from one workflow language to another. There are several projects that try to solve this task.

Seqera AI is a commercial tool that is able to convert CWL or WDL to Nextflow and vice-versa using Claude LLM. Seqera AI provides translation with step by step explanation and analysis of the script. It is also able to run generated Nextflow, find errors and automatically fix it. Seqera AI is a great tool for generating and modifying pipelines, studying Nextflow and running it with uploaded input files. However as an LLM it has certain limitations. It works without a clear algorithm of pipeline generation. Sometimes generated workflows are overcomplicated. Sometimes LLM hallucinates and produces scripts with errors which can not be easily traced. It also can get into never ending self fixing cycles. According to information at https://seqera.io/user may use Seqera AI freely with 250 chat sessions per month. Generation of even the simplest pipeline takes some time as requests should be passed to LLM.

CNT ¹ is a semiautomated translator converting CWL workflows into Nextflow ones. It integrates tool-level conversion, graph dependency analysis, and correctness checks to provide highly automated translation coverage, significantly reducing the development time while satisfying language-specific requirements like building a proper dataflow model when creating workflows. According to the authors estimate it can cover up to 81% of the original workflows. ¹

Janis ²³ is a framework creating specialised, simple workflow definitions that are then transpiled to CWL or WDL. Conversion to Nextflow is currently under progress.

cwl2nxf ²⁴ is a prototype conversion tool from CWL to Nextflow. However it is abandoned now.

Since 2001 we have been developing the BioUML ²⁵ platform - an integrated environment for systems biology and collaborative analysis of biomedical data. Like the Galaxy platform it provides a graphical web interface for workflow visualisation, creation and editing. geneXplain platform is a commercial version of BioUML platform and provides thousands of ready workflows for analysis of different omics data.

BioUML uses its own format for workflow storage and engine for workflow execution. BioUML is also integrated with Galaxy, cwlTool, cromwell and Nextflow for execution of workflows in Galaxy, CWL and Nextflow formats correspondingly. However this integration is quite difficult and is not seamless.

So now we are designing a new architecture where workflows in different formats can be converted into a common object-oriented model presented by a set of Java classes. This model can be visualised and edited as a graph. Then workflow text in different formats can be generated from this model.

As a first step of this project we describe here mapping of WDL workflows into a common workflow model, editing graphs visually and generation of WDL and Nextflow scripts from this model.

We have split corresponding parts of code from the BioUML project into an independent project on GitHub: https://github.com/genespace-ru/workflow-engine . We hope that the created command line application, web application and web service will be useful for many users and can be used as parts of other projects related to workflows.

While both WDL and Nextflow describe pipelines of interconnected tools, their approaches are very different which makes conversion between them harder. -

WDL uses intuitive syntax and a static structure of workflow with explicit cycles (scatters), arrays, tasks, inputs and outputs.

Our aim was to develop a set of universal rules for generating Nextflow scripts based on the data model created from WDL formalism (tasks, expressions, cycles and conditional blocks).

Description of tools connected into workflow (tasks in WDL and processes in Nextflow) are very similar in both languages and have the same structure with inputs, command, outputs and metadata, thus conversion is quite straightforward, with small changes to executing command itself necessary when WDL idioms are used (e.g. ~{sep=“ ” files} -> files.join()).

The main difficulty in conversion between WDL in Nextflow is dealing with arrays and cycles. WDL allows for very intuitive data manipulation while Nextflow tends to more idiomatic structures.

In Nextflow channels are executed asynchronously as elements are delivered via channel and order of outputs is not determined by order of inputs. To emulate WDL behavior of a determined output array we used a “fair true” directive for each process which forces output order to be the same as input order. Netflow also has arrays and in some cases WDL arrays are translated to Nextflow arrays while in others to Nextflow channels. In order to facilitate work with them we defined support Groovy functions such as get (array, i). length (array) that can work both with arrays and channels. In some cases we need to be sure that we work with channels (e.g. running call in cycle for elements of array) to that end we use the function to Channel (array) which returns a channel in any case (input can be both array or channel).

WDL defines a list of prebuilt functions such as “basename”, ”sub”, “read_int” and so on. To ease conversion we have created Groovy functions with the same name and functionality. All those functions are stored in supplemented biouml_function.nf script which is included into Nextflow converted from WDL. This allows us to minimally change expressions during conversion (see Table 3).

A full table of conversion rules and defined Groovy functions is presented at workflow-engine.readthedocs.io.

We have tested our approach on a limited number of tests for conversion of WDL scripts into Nextlow ones. So it is possible that there are cases where incorrect Nextflow will be generated. We will continue our work to create a more powerful test suite. Also some WDL workflows can be translated in a more concise way using Nextflow’s more complex structures.