Python in an HPC Environment

Presenters: Ryan Mokos, Roland Haas, and Maxim Belkin, NCSA, University of Illinois

Tuesday, March 16, 2021




There is no denying that Python has become ubiquitous. The power of the language, combined with its relative ease of use and the extensive list of packages that have been developed for it, have quickly made it a favorite of many researchers. But while it may be fairly easy to produce working python code, it is not always so easy to write python code that runs fast and efficiently. This is particularly true in High Performance Computing (HPC) environments, where some workflows do not scale well when run on hundreds or thousands of processors. In this presentation, we will discuss techniques for parallelizing python computation and IO in HPC environments, with a particular emphasis on running python codes on the Blue Waters supercomputer. We will cover the configuration of the Blue Waters python environment, how to install packages, and touch on the role that containers can play in a programmer’s workflow.

Prerequisites: Participants are expected to have familiarity with using Python. No prior HPC experience is needed.