Cloud Resource Federation for Galaxy

Presenter: Enis Afgan, Research Scientist, Taylor lab, Johns Hopkins University

Date: January 23, 2019

Slides: https://uofi.box.com/s/tpn2g4yu6ak2j8p2d5fk1xurtpza7wpm

Video: https://youtu.be/1QCsckC8vHg

Abstract

The popularity of Galaxy (https://galaxyproject.org) is calling for extensive compute and storage capacity to be made available. The Galaxy Project and the community at large have been making strides in this effort by creating a variety of resources available. These include free public servers, virtual machines, cloud instances, usegalaxy.* federation, container images, etc. However, such dedicated installations of Galaxy instances is gradually leading to many silos of data, shared objects, tools, and expertise. In order to further strengthen the community and help build on each other’s contributions, we are actively working on the notion of resource federation for Galaxy. This means that it will be possible to bring together disparate data and compute resources to effectively handle large data and long computations. In this talk, we will present the motivation for the above problem, the ongoing efforts, and present a method for dynamically linking external compute resources from a variety of cloud resources to a Galaxy server. Much of the presented software stack and services are not specific to Galaxy and can also be used by other science gateways.

Target audience: IT professionals (admins, DevOps, developers).
Prerequisites: None.
Reference materials: None.

Biography

Enis Afgan is a research scientist in the Taylor lab at Johns Hopkins University working on the Galaxy project. For the past decade, he has been evolving the capacity of cloud computing resources to create an accessible platform for processing biological data. He has been a chief architect of a number of popular software solutions we will hear about today, including CloudMan, CloudBridge, CloudLaunch, and the Genomics Virtual Laboratory. More broadly, his interests focus around accessibility of distributed computing systems. He obtained his Ph.D. from the University of Alabama at Birmingham in 2009 and has published over 50 articles..