We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Playing with Nix in adverse HPC environments

Formal Metadata

Title
Playing with Nix in adverse HPC environments
Title of Series
Number of Parts
542
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
What happens when you have access to large clusters, but have little control over the software installed on the machines? Unfortunately, this is the current scenario that researchers often find in HPC clusters, which include very old software stack, a brittle environment and non-cooperative sysadmins. We have been experimenting with Nix to provide an up-to-date system running on top of the already existing software, without root permissions with the help of user namespaces. In this talk we give a tour on the problems we found and how we solved them: Starting from the installation and configuration of nix to be used by multiple users when we lack a shared /nix store. - Avoiding library contamination from /usr/lib with an isolated root mount - Interactive development while compiling the code inside the isolated environment with a patched nix-portable - Adding custom compilers to the stdenv Building packages tuned to an specific CPU with vectorization optimizations in mind. - Running the benchmarks with SLURM inside the isolated environment with multiple compute nodes. Improving MPI fast zero-copy transfer inside user namespaces.