High-performance workflows with Swift/T

Presenter: Justin Wozniak, Computer Scientist, Argonne National Laboratory, University of Chicago

Date: September 20, 2017

Video: ttps://www.youtube.com/watch?v=BWRpboPBArI

Abstract

Swift/T is a workflow system that enables users to combine traditional workflow features like the execution of external programs with calls to in-memory functions. Swift/T runs as an MPI program managed by a scalable load balancer, allowing it to handle trillions of tasks on large supercomputers. It offers a concise high-level language for describing data dependencies and iterations. It also allows workflows to call into embedded scripting interpreters, such as Python, R, and JVM scripting languages, allowing scripts and systems (e.g., Numpy) in these languages to be applied at large scale. In this demo, we will demonstrate Swift/T usage with Python functions on Blue Waters.

Target audience: Researchers, developers, and scientific teams.

Prerequisites: None.

Training and reference materials: