Experiences of supporting HPC/HTC workloads on private cloud resources, withideas for how to do this better and description of trends for non-traditionalHPC resource provision. I will discuss my work as part of the Operations Team for the eMedLab privatecloud, https://www.emedlab.ac.uk, which is a large-scale (6000-core, 5PB)biomedical research cloud using HPC hardware, aiming to support HPC workloads.This includes an account of the technical challenges of marrying traditionalHPC components such as parallel filesystems with the rather differentexpectations of the OpenStack cloud management framework. Other challengesfaced include educating users about the basics of virtual environments,providing access to restricted-license large-scale (800TB) datasets in amulti-tenant environment, and achieving good performance inside virtual HPCclusters. Looking to the future, container orchestration engines such as Kubernetesprovide a very promising approach, as the containerisation of scientificapplications gains momentum, and I will describe existing and future work inthis area. |