Buildtest is an HPC testing framework to aid HPC facilities to perform acceptance testing for their system. HPC systems are growing in complexity, with a tightly coupled software and system stack that requires a degree of automation and continuous testing. In the past decade, two build frameworks (Spack, EasyBuild) have emerged and widely used in HPC community for automating build & installation process for scientific software. On the contrary, testing frameworks for HPC systems are limited to a few handful (ReFrame, Pavilion2, buildtest) that are in active development. In buildtest, users will write test recipes in YAML called buildspecs that buildtest process to generate a shell script. buildtest utilizes versioned-based JSON Schema for validating buildspecs and currently, we support two main schemas (compiler, script). The script schema and compiler schema are used for writing traditional shell-scripts (bash, sh, csh), python-scripts and single source compilation test. In this talk we will present an overview of buildtest and how one can write buildspecs. Furthermore, we will discuss Cori Testsuite in buildtest with several real examples on testing various components for Cori system at NERSC. |