We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

How the Spack package manager tames the stat storm

Formal Metadata

Title
How the Spack package manager tames the stat storm
Title of Series
Number of Parts
542
Author
Contributors
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
In high performance computing, package managers like Nix, Guix, Gentoo Prefix, and Spack are used to install applications and their dependencies. They allow multiple variants and flavors of the same package to coexist, by installing every package into a unique prefix directory that embeds a hash derived from the dependency graph. This non-standard directory structure can result in increased startup time of executables, as the dynamic linker has to search through many directories to locate all required libraries. This is especially problematic in the context of HPC and shared filesystems, since many instances of the same executable may start in parallel, causing a "stat storm" on the typically slow filesystem. This talk explores benefits and drawbacks of different solutions, and shows in particular how the Spack package manager solves this problem.