The advent of high-throughput sequencing technologies - accompanied with the recent advances in open source software tools, open access data sources, and cloud computing platforms - has enabled the genomics community to develop and use sophisticated application workflows. Such workflows start with voluminous raw sequences and end with detailed structural, functional, and evolutionary results. The workflows involve the use of multiple software tools and data resources in a staged fashion, with the output of one tool being passed as input to the next. As one example, a personalized medicine workflow 123 based on Next Generation Sequencing (NGS) technology can start with short DNA sequences (reads) of an individual human genome and end with a diagnostic and prognostic report, or potentially even with a treatment plan if clinical data were available. This workflow involves the use of multiple software tools to assess the quality of the reads, to map them to a reference human genome, to identify the sequence variations, to query databases for the sake of associating variations to diseases, and to check for novel variants. As another example, consider a workflow in the area of metagenomics 4567. Such workflow can start with a large collection of sequenced reads and end up with determination of the existing micro-organisms in the environmental sample and an estimation of their relative abundance. This workflow also involves different tasks and software tools, such as those used for assessing the quality of the reads, assembling them into longer DNA segments, querying them against different databases, and conducting phylogenetic and taxonomical analyses.

To simplify the design and execution of complex bioinformatics workflows, especially those that use multiple software tools and data resources, a number of scientific workflow systems have been developed over the past decade. Examples include Taverna 89, Kepler 10, Triana 1112, Galaxy 13, Conveyor 14 Pegasus 15, Pegasys 16, Gene Pattern 1718, Discovery Net 1920, and OMII-BPEL 21; see 22 for a survey and comparison of some of these tools.

All such workflow systems typically adopt an abstract representation of a workflow in the form of a directed graph, where nodes represent tasks to be executed and edges represent either data flow or execution dependencies between different tasks. Based on this abstraction, and through a visual front-end, the user can intuitively build and modify complex applications with little or no programming expertise. The workflow system maps the edges and nodes in the graph to real data and software components. The workflow engine (also called execution or enactment engine) executes the software components either locally on the user machine or remotely at distributed locations. The engine takes care of data transfer between the nodes and can also exploit the use of high performance computing architectures, if available, so that independent tasks run in parallel. This makes the application scientist focus on the logic of their applications and no longer worry about the technical details of invoking the software components or use of distributed computing resources.

Within the bioinformatics community, two workflow systems have gained increasing popularity, as reflected by their large and growing user communities. These are Galaxy 13 and Taverna 89. Both systems are efficient, open source, and satisfy to a great extent the requirements of the bioinformatics community. Taverna has been developed primarily to simplify the development of workflows that access and use analyses tasks deployed as remote web and grid services. It comes with an associated directory of popular remote bioinformatics services and provides an environment that coordinates their invocation and execution. Galaxy has been developed primarily to facilitate the execution of software tools on local (high performance computing) infrastructure while still simplifying access to data held on remote biological resources. Its installation includes a large library of tools and pre-made scripts for data processing. Both systems are extensible, allowing their users to integrate new services and tools easily. Each system offers log files to capture the history of experiment details. Furthermore, both systems provide web-portals allowing users to share and publish their workflows: These are the myExperiment portal for Taverna 2324 and the Public Pages for Galaxy 13. The features of both systems are continuously being updated by their development teams and their user communities are active in developing and sharing new application workflows.

However, since both Taverna and Galaxy have been developed with different use cases and execution environments in mind, each system tends to be suited to different styles of bioinformatics applications. The key differences between the two systems can be categorized into three major classes:

1. Execution environment and system design: Taverna is oriented towards using web-services for invoking remote applications, while Galaxy is oriented towards efficient execution on a local infrastructures.

2. Model of computation: Taverna includes control constructs such as conditionals and iterations, and data constructs that can handle (nested) lists (in parallel) using a number of pre-defined operations. These constructs are not directly available in Galaxy, which puts a limitation on the types of workflows that can be executed on Galaxy.

3. Workflow description language: Taverna uses the XML-based language SCUFL for describing the workflows, while Galaxy expresses workflows in its own language using JSON format.

These differences lead to two major consequences: First, some tasks can be implemented easily on one system but would be difficult to implement on the other without considerable programming effort. Second, a (sub-) workflow developed on one system cannot be imported and re-used by the other easily (i.e., lack of interoperability), which limits sharing of workflows between their communities and leads to duplication of development efforts.

In this paper, we present Tavaxy, a pattern-based workflow system that can integrate the use and execution of Taverna and Galaxy workflows in a single environment. The focus of Tavaxy is facilitating the efficient execution of sequence analysis tasks on high performance computing infrastructures and cloud computing systems. Tavaxy builds on the features of Taverna or Galaxy providing the following benefits:

· Single entry point: Tavaxy is a standalone pattern-based workflow system providing an extensible set of patterns, and allows easy integration with other workflow systems. It provides a single environment to open, edit, and execute its own workflows as well as integrate native Taverna and Galaxy whole- or sub-workflows, thus enabling users to compose hybrid workflows. Figure 1 summarizes the different integration use cases at run-time and design-time levels in Tavaxy. (The replacement of remote calls with local tools is addressed in the next paragraph. The computation of maximal external sub-workflows is a performance optimization step discussed later in this paper.)

· Transparent use of local and remote resources: For most programs, Tavaxy allows its user to choose whether a task should run on local or remote computational resources. Furthermore, if a Taverna workflow is imported (e.g., from my Experiment), Tavaxy offers users an option to replace calls to remote web services automatically with calls to corresponding tools that run on a local computing infrastructure, or vice versa. (Note that almost all the workflows published on myExperiment are based on using remote services). Changing the default mode of invocation in either Taverna or Galaxy requires programming knowledge, and it is difficult to achieve by the non-programming scientist.

· Simplified control and data constructs: Tavaxy supports a set of advanced control constructs (e.g., conditionals and iterations) and data constructs (e.g., nested lists) and allows their execution on the local or remote computational infrastructures. The use of these constructs, which are referred to as “patterns” in Tavaxy, facilitates the design of workflows and enables further parallelization, where the data items passed to a node can be processed in parallel. The user of Tavaxy has the extra advantages of 1) adding these constructs to imported Galaxy workflows, and 2) using these constructs on the local infrastructures; features that are available only in Taverna and only for remote tools.

Beyond these integration issues, Tavaxy provides the following additional features that facilitate authoring and execution of workflows:

· Enhanced usability: Tavaxy uses flowchart-like elements to represent control and data constructs. The workflow nodes are annotated with icons to reflect if they are executed locally or remotely. The tool parameters can be defined either at the design- or run-time of the workflow. The data patterns offered in Tavaxy further facilitate the composition of workflows, making them more compact, and enable exploitation of local high performance computing infrastructure without any additional effort. Furthermore, each user has an account associated with its data and each workflow is further associated with its history as well as previously used datasets within the user account.

· Modularity: Tavaxy is modular; it separates the workflow composition and management modules from the workflow engine. Its workflow engine is a standalone application accepting both workflow definitions and data as input. This feature, as will be made clear later in the manuscript, is of crucial importance for implementing control constructs and for supporting cloud computing.

· High performance computing infrastructure support: Tavaxy can readily run on a computer cluster, once a job scheduler system (like PBS Torque or SGE) and a distributed file system (like NFS) are installed. The execution of parallel tasks is handled automatically by the workflow engine, hiding all invocation details.

· Cloud computing support: Tavaxy is cloud computing friendly, enabling users to scale-up their computational infrastructure on a pay-as-you go basis, with reduced configuration efforts. Through a simple interface within the Tavaxy environment, a user who has a cloud computing account (e.g., at the Amazon AWS platform) can easily instantiate the whole system on the cloud, or alternatively use a mixed mode where his local version of the system can delegate the execution of a sub-workflow or a single task to a Tavaxy cloud instance.

In the remaining part of this section, we will review basic concepts of workflow interoperability and workflow patterns that contributed to the design and development of Tavaxy.

Tavaxy workflows are directed acyclic graphs (DAGs), where nodes represent computational tools and edges correspond to data flows or dependencies between them. The workflow patterns defined and used in Tavaxy have special meanings in this DAG, as will be explained in detail later in the pattern implementation subsection. The Tavaxy engine is based on a data flow model of execution 22283031323334, in which each node (task) can start computation only once all its input data are available. The Tavaxy workflow engine is responsible for keeping track of the status of each node and for invoking its execution when the predecessor nodes have finished execution and when its input data is completely available. When executing on a single processor, where tasks are executed sequentially, the order of task invocation can be determined in advance. This is achieved by traversing the DAG and scheduling a node (i.e., adding it to the ready queue) only if all the its predecessor nodes are already scheduled. As such, this scheduling is referred to as a static scheduling 34. When executing on multiple processors, independent ready tasks can be executed in parallel. In this case, the Tavaxy engine keeps looking for ready tasks and launches them concurrently. On multi-core machines, the engine uses multi-threading to handle the execution of concurrent tasks. On a computer cluster, it passes the concurrent tasks to a job-scheduler, which in turn distributes them for execution on different cluster nodes. The default job-scheduler used in Tavaxy is PBS Torque, and it is set-up over a shared file system (NFS for local setting and S3 for cloud infrastructure) to guarantee availability of data for all cluster nodes.

A Tavaxy workflow is defined and stored in tSCUFL format, which is similar in flavor to the Taverna SCUFL format. However, there are two main differences between the two formats: 1) A node’s parameters are represented in tSCUFL by default as attributes of the respective tool, whereas they are considered as input items in SCUFL. 2) The workflow patterns (e.g., conditionals and iteration) are explicitly specified in tSCUFL but implicitly defined in SCUFL.

Tavaxy provides an easy-to-use environment allowing the execution of Tavaxy workflows that integrate Taverna and Galaxy workflows as sub-workflows. Such integration can be achieved at both design-time and run-time:

For run-time integration, Tavaxy can execute both Galaxy and Taverna (sub-) workflows ‘as is’, with no modification. For Galaxy workflows, this is straightforward, because the Tavaxy engine is compatible with the Galaxy engine and follows the same model of computation. For Taverna workflows, Tavaxy can execute a Taverna (sub-) workflow by invoking the Taverna engine through a command line interface that takes both the Taverna (sub-) workflow file and its data as input. The Tavaxy mapper component assures the correct data transfer between the Taverna engine and other nodes. This is achieved by setting source and destination directories and input/output file names in appropriate manners.

For design-time integration, Tavaxy imports and manipulates workflows written in either Galaxy or Taverna formats. Tavaxy can import a Galaxy workflow file to its environment, allowing its modification and execution. The engineering work for this step includes translation of the JSON objects of the Galaxy workflow to the tSCUFL format of Tavaxy. For Taverna workflows, the implementation addresses the differences in the model of computation and workflow languages. Specifically, the workflow engine of Tavaxy is a data-flow oriented one, with no explicit specification of control constructs, while the Taverna engine supports both data- and control-flow constructs.

The Taverna workflow language is SCUFL/t2flow but that of Tavaxy is tSCUFL. To overcome these differences, we use the concept of workflow patterns to 1) execute (“simulate”) the execution of Taverna control and data constructs on the data-driven workflow engine of Tavaxy; and 2) to provide a pragmatic solution to language translation where a Taverna (sub-) workflow is decomposed into a set of patterns that are then re-written in Tavaxy format. The following section introduces the Tavaxy workflow patterns and their implementation.

We divide the Tavaxy workflow patterns into two groups: control patterns and data patterns. In the remainder of this subsection, we define these patterns and their implementation on the Tavaxy data-flow engine.

Figure 3 (left) shows the architecture of Tavaxy, which is composed of four main components: 1) workflow authoring module, 2) workflow pattern database, 3) workflow mapper, and 4) workflow engine. On top of these components, we developed user accounts to maintain users workflows and data. We also developed a repository of public workflows that is shared between users. Figure 3 (upper right) shows the main Tavaxy page containing links to different system parts and utilities.

As briefly mentioned before in the introduction, we provide three modes for using cloud computing: 1) whole system instantiation, 2) sub-workflow instantiation, and 3) tool (service) instantiation. To further simplify the use of the first mode, we installed an instance of Tavaxy (including the whole web-interface and tools) on an Amazon AWS virtual machine and deposited a public image of it at the Amazon web-site. A user who has an Amazon account can directly start the image and use it. Based on Amazon APIs, this image can establish a computer cluster upon its activation. The user can specifically define the type of nodes (e.g., large or extra large) and their number. The Amazon S3 storage is used as a shared storage for the computer cluster. We developed several interface functions that manage data transfer among the compute nodes and the shared storage of the cluster at run time. Figure 4(left) shows a screen shot of the Tavaxy interface page, where the user can configure the cluster and storage.

In the second mode, the user already has a Tavaxy version installed on his local machines (called local Tavaxy) and delegates the execution of one or multiple sub-workflows to be executed on the cloud. To support this scenario, a lightweight version of Tavaxy has been deposited at the Amazon platform as a virtual machine image. From a simple user interface in the local Tavaxy, the user can start and configure a cloud cluster using the prepared Tavaxy image.

At run-time, the local version of Tavaxy communicates with the cloud counterpart, using a simple asynchronous protocol (similar to the REST protocol), to send the sub-workflow, execute it, and retrieve the results. The input and output data related to such a sub-workflow flow according to one of two scenarios:

1. The input data is sent to the mounted disk of the main cloud machine along with the workflow to be executed. After processing, the output is sent back to the local Tavaxy. After termination of the machine, the input and result data are lost, unless they are moved by the user to a persistent storage. This scenario is useful in case no computer cluster is needed.

2. The input data is sent to a shared volume in the persistent S3 storage (this can be done offline), where the compute nodes of the computer cluster can easily access it. Because reads and writes to S3 require the use of Amazon APIs, we developed special scripts to facilitate this access between the local Tavaxy and S3 on one side and between the compute nodes and S3 on the other side. After execution of the sub-workflow, a copy of the output is maintained on the S3 and another copy is sent to the local Tavaxy to complete the execution of the main workflow.

The third mode is a special case of the second mode, where the user can delegate the execution of only a single task to the cloud. For this mode, we also use a simple protocol to manage the data transfer and remote execution of the task on the cloud. Figure 4(right) shows the architecture of the cloud version of Tavaxy and the data flows among its components.

There are different ways to access and use the Tavaxy system from its main home page:

1. Downloadable version: The whole Tavaxy system, with all features described in this manuscript, can be downloaded for local use. The bioinformatics packages are provided in a separate compressed folder, because we assume that some users already have installed the packages of interest on their local systems and just need the Tavaxy system. The packages currently include about 160 open source tools, coming from EMBOSS 35, SAMtools 36, fastx 37, NCBI BLAST Toolkit 383940, and other individual sequence analysis programs. Addition of extra tools is explained in the Tavaxy manual.

2. Web-based access: We offer a traditional web-based interface to a Tavaxy instance for running small and moderate size jobs. For large scale jobs, we recommend the use of cloud version.

3. Cloud-computing based access: In this mode, each user creates a Tavaxy instance with the hardware configuration of choice on the AWS cloud. The interesting feature in this model is that multiple users may have multiple Tavaxy systems, each with different configuration (number and type of ‘virtual’ machines). The Tavaxy instances on the cloud already include the 160 tools currently tested. They also include a number of databases to be used with the cloud machines, such as the NCBI (nucleotide and protein) and swissprot databases.

At the time of preparing this manuscript (June 2011), the Taverna repository myExperiment contained 557 workflows in SCUFL (Taverna1) format and 554 workflows in t2flow format (Taverna2). By manual inspection, we found that 296 workflows (96 in SCUFL format and 200 in t2flows format) are related to the sequence analysis domain, which is the main focus of this version of Tavaxy. To help the community, we already imported all these workflows into the Tavaxy environment, and arranged them in a special web-accessible repository for public use. We also provided the user with optimized versions of the sequence analysis workflows, where many of the web-services are replaced with local invocations of the corresponding local tools distributed with Tavaxy. We also imported all public Galaxy workflows from the Galaxy Public Pages and added them to this repository. The workflows imported from both the Taverna and Galaxy repositories are included in the Tavaxy system, and will be kept up-to-date on its web-site. These workflows can serve as “design patterns” that can can be used to speed up workflow development cycle, when developing more complex workflows.

In the following sub-sections, we introduce two case studies that demonstrate the key features of Tavaxy. In the first case study, we demonstrate 1) how Taverna, Galaxy, and Tavaxy sub-workflows can be integrated in a single Tavaxy workflow, highlighting both the integration capabilities and use of workflow patterns; and 2) the optimization steps included before the execution of imported workflows and their effects on the performance of the system. In the second case study, we demonstrate 1) the use of Tavaxy for a metagenomics workflow based on NGS data; 2) the advantages of using advanced data patterns in facilitating the workflow design and supporting parallel execution; 3) the speed-up achieved by using local HPC infrastructure; and finally 4) the efficient and cost-saving use of cloud computing.

Figure 5 shows a workflow for finding homologous protein sequences and analyzing them. The workflow starts with reading a DNA/protein sequence from the user. If the input is a DNA sequence, it is translated to a protein sequence. The input sequence is passed to BLAST 3839 to find similar sequences. The output of BLAST, which is a list of Genbank IDs, is then compared to a user-provided list of sequence IDs to exclude common sequences from the output list. The protein sequences of the exclusive IDs are then retrieved and passed to the programs ClustalW 4142 and MUSCLE 43 for computing multiple alignment. ClustalW is also used for computing a phylogenetic tree.

Searching the myExperiment repository, there is already an existing Taverna implementation for most of the desired workflow, deposited under the name “workflow_for_protein_sequence_analysis” 44, and Figure 6 shows its implementation as it appears in the Taverna authoring module. The missing functionality in this Taverna workflow are the two parts highlighted in Figure 5, including the parts for translating the DNA sequence into protein sequence and the one for MUSCLE-consensus. In the original Taverna implementation, the software tools BLAST, ClustalW, and phylogeny plotting are invoked through web-service interfaces. The other intermediate steps are executed by built-in Taverna programs.

We downloaded the Taverna workflow and imported it into Tavaxy; Figure 7 shows the same workflow in the Tavaxy environment. At this step, the user may choose to execute this workflow as it is from Tavaxy, or may choose to optimize the execution of the workflow and/or customize it by adding further tasks. For example, for this workflow, the user can replace web-services with equivalent locally installed tools through a simple user interface. The workflow mapper carries out this replacement and can, according to user choices, coalesce the remaining Taverna tasks into maximal sub-workflows, as described earlier in the Tavaxy implementation section. In this example, we decided that the ClustalW and the phylogeny analysis parts of the workflow run on the local infrastructure, while the BLAST part still runs remotely. Figure 8 shows the optimized version of this workflow, where the maximal Taverna sub-workflows are computed. The functionality of this workflow can be augmented with further tasks. First, we re-used a native Galaxy (sub-) workflow that computes multiple alignment using the MUSCLE program and computes the consensus sequence. Second, we added a Tavaxy sub-workflow, in which the DNA sequences are translated into protein sequences, instead of ignoring processing them. To link the translated sequences to the other parts of the workflow for further analysis, the data merge pattern is used to pass the protein sequences. These extra parts are highlighted in Figures 5 and 8.

Figure 9 (left) shows a flow chart representation of a metagenomics workflow deposited on the Galaxy public pages 4647. The input to this workflow is a set of NGS reads and associated quality data. The workflow starts with quality check of the reads and computation of their lengths. The high quality reads are queried, using the MegaBLASTtool 40, against two different databases chosen by the user. The reads with good alignment coverage are retained for further analysis. Finally, the taxonomical information of the successful reads are extracted from the alignment file and a taxonomy tree is plotted.

In the original implementation of this workflow on Galaxy 4647, and as depicted in the schematic representation of Figure 9, we can identify two issues: First, the input reads are passed to MegaBLAST is a single multi-FASTA file which implies sequential processing of the queries against the database. Second, there are two nodes for MegaBLAST: one to consider the NCBI_WGS database and the other to consider the NCBI_NT database. To query more databases in Galaxy, additional nodes should be manually added; this will yield a bulky workflow for a large number of databases. In Tavaxy, we can enhance the design and execution of this workflow with respect to these two issues.

For the first issue, we use the Tavaxy list pattern in association with MegaBLAST so that the input multi-FASTA file is handled as a list of items. This will immediately lead to parallelization of this step. A list item could be a single FASTA sequence or a block of multiple FASTA sequences. We recommend that the input reads are divided into a list of n blocks, each of size k sequences. The parameter k is set by the user and it should be proportional to the number of processors available. (The list is defined by a special node and its items (blocks) are separated by a special user-defined symbol.) When the workflow with the list pattern is executed, multiple versions of MegaBLAST will be invoked to handle these blocks in parallel.

For the second issue, concerning the simple integration of more databases, we will use only just one MegaBLAST node and create a list of input databases. This list is passed as input to the MegaBLAST node. To ensure that each read is queried against all given databases, we use the cross product operation defined over the list of databases and the list of input sequences. For m databases and n blocks, we have n × m invocations of MegaBLAST, which can be handled in parallel without extra coding effort.

Figure 9 (right) shows a schematic representation of the enhanced workflow with the list pattern. Figure 10 shows the implementation of the enhanced workflow in Tavaxy. In this figure, the special node “split_into_list” defines the list items from the multi-FASTA file.