Getting started with WDL and Cromwell for workflow management on multiple platforms

Presenters: Geraldine Van der Auwera, PhD, Associate Director of Outreach and Communications, Data Sciences Platform, Broad Institute of MIT and Harvard; and Ruchi Munshi, Senior Software Product Manager, Data Sciences Platform, Broad Institute of MIT and Harvard

Date: November 21, 2018

Slides: https://uofi.box.com/s/tjqap06qc8rji0ux1p10jajbtu33lj9i

Video: https://www.youtube.com/watch?v=mFzfeDTnDSk

Abstract

Cromwell is a Workflow Management System geared towards executing scientific workflows. It can be run on multiple platforms including popular on-premises schedulers such as SGE, Slurm and LSF, as well as public clouds such as Google Cloud Platform (GCP), Amazon Web Services (AWS) and Alibaba Cloud. The main advantage of using Cromwell is that it it simple to integrate with various platforms, and can be used to selectively dispatch work among different platforms.

Cromwell takes as input a workflow defined using either the Workflow Description Language (WDL) or Common Workflow Language (CWL). While both CWL and WDL allow for portable specification of scientific workflows for processing data, WDL has an additional goal of  providing a human-readable and writable syntax, making it straightforward to define analysis tasks, chain them together in workflows, and parallelize their execution.

In this webinar, we will demonstrate the basics of writing workflows in WDL and executing them on Cromwell on a local system and on cloud.

Target audience: Researchers and bioinformaticians interested in getting started with WDL and Cromwell for authoring and executing scientific workflows. The demonstration will center on use cases in the life sciences but will be applicable to other sciences.

Prerequisites: None.