Getting I/O Done with Parallel HDF5 on Blue Waters

Presenters: Frank Willmore, Data Specialist, HDF Group; and Gerd Heber, Applications Architect, HDF Group

Date: March, 2018




It is part of The HDF Group’s mission to “help people understand HDF and make effective use of its capabilities, to ensure long-term access to HDF data, and to advance the state of the art in data management”. However, the proof of the pudding is in the eating. This is a retrospective on what we, The HDF Group and a few daredevils (science teams), have learned about the use of parallel HDF5 in the “real world” (Blue Waters).

Target Audience: Researchers and developers, data scientists

Prerequisites: General scientific/high-performance computing background

Training and Reference Materials:


Frank Willmore completed his PhD in chemical engineering in 2006 at the University of Texas under the direction of Isaac C Sanchez, creating computer models to study the role of free volume in small molecule diffusion. Frank is an active member of the American Institute of Chemical Engineers and continues to organize sessions around computing at their annual meeting. A proud alum of the Texas Advanced Computing Center (TACC), he is the lead editor and a contributing author to “Introduction to Scientific and Technical Computing” (CRC Press) based on the for-credit course he taught for them at the University of Texas. Frank works as a developer / Scientific Data Specialist with The HDF Group.