DolphinNext: a distributed data processing platform for high throughput genomics

BMC Genomics

Table 1 Comparison of related applications

	DolphinNext	Galaxy [5]	Sequanix [10]	Taverna [9]	Arvados [63]
Platform^a	JS/PHP	Python	Python	Java	Go
Workflow management system	Nextflow	Galaxy	Snakemake	Taverna	Arvados
Native task support^b	Yes (any)	No	Yes (bash only)	Yes (bash only)	Yes
Common workflow language^c	No	Yes	No	No	Yes
Streaming processing^d	Yes	No	No	No	Yes
Code sharing integration^e	Yes	No	No	No	Yes (GitHub)
Workflow modules^f	Yes	Yes	Yes	Yes	Yes
Workflow versioning^g	Yes	Yes	No	No	No
Automatic error failover^h	Yes	Yes	No	No	Yes
Nested workflows	Yes	Yes	No	Yes	No
Used syntax/ semantics	own/own	XML/own	Python/own	own/own	Python/own
Web-based	Yes	Yes	No	No	No
Web-based process developmentⁱ	Yes	No	No	No	No
Distributed pipeline execution^j	Yes	No	No	No	No
Container Support
Docker support	Yes	Yes	Yes	No	Yes
Singularity support	Yes	Yes	Yes	No	No
Built-in batch schedulers
LSF	Yes (Native)	Yes (DRMAA)	Yes (Native)	No	No
SGE	Yes (Native)	Yes (DRMAA)	Yes (Native)	Yes (Native)	No
SLURM	Yes (Native)	Yes (DRMAA)	Yes (Native)	No	Yes (Native)
IGNITE	Yes (Native)	No	No	No	No
Built-in cloud
AWS (Amazon Web Services)	Yes	Yes	No	Yes	Yes
GCP (Google Cloud Platform)	Yes	Yes (Partial)^k	No	No	Yes
Autoscaling	Yes	Yes	No	No	Yes

^aThe technology and the programming language in which each framework is implemented
^bThe ability of the framework to support the execution of native commands and scripts without re-implementation of the original processes
^cSupport for the CWL specification
^dAbility to process tasks inputs/outputs as a stream of data
^eSupport for code management and sharing platforms, such as GitHub
^fSupport for modules, sub-workflows or workflow compositions
^gAbility to track pipeline changes and to execute different versions at any point in time
^hSupport for automatic error handling and resume execution mechanism
ⁱAbility to add new processes in an embedded web editor without a wrapper or any installation of the wrapper
^jSupport for executing the same pipeline without any change in multiple computing environments to process the data within a single interface (e.g. hpc clusters, a workstation and cloud)
^kA Galaxy instance can be launched in Google cloud but for one-time use. When it is shut down, they are permanently deleted

ISSN: 1471-2164