Spack is an open-source HPC package manager that simplifies building applications from source code. It has built in dependency resolution and makes it easy to manage multiple build stacks with differing build specs, compilers, and MPI flavors.
Using Installed Packages
Packages on Fluid-Slurm-GCP clusters are installed by Spack and are exposed to users through lmod. Our solutions come with a preconfigured sitewide packages.yaml, compilers.yaml, and modules.yaml in addition to GCC/10.2, OpenMPI/4.0.2, and Singularity/3.7. All of these packages are managed via Spack. Additionally, fluid-slurm-gcp comes with an installation of the oneAPI™ HPC Toolkit from Intel, which can be brought into your path using environment modules.
To show what packages are currently available on the cluster, you can use the module list command.
Loading and Unloading Packages
To load packages that have been previously installed by spack, you can use the module load command.
For example, to load OpenMPI/4.0.2 compiled with the GCC/10.2.0 compiler, you can use the command
If you want to remove a package from your path, you can use module unload command.
For example, to unload OpenMPI/4.0.2, you can use the command
Install additional packages with Spack
Spack supports the installation of a large number of packages. All packages are compiled from source, giving you the ability to control compilation flags and the types of features to include with each packages build.
Use the checklist below to help guide package installation with spack.
First check that the desired package can be built with spack using the spack list command. Alternatively, you can visit the spack packages list page to see if the desired package is supported.
Use the spack info command to familiarize yourself with the packages dependencies, available versions, and available build specs.
Preview what packages will be installed by using spack spec. Keep in mind that you can specify the package version and the compiler to build with. Additionally, you can concretize the package variants to customize the installation. For example, some packages like hdf5 can be built with or without mpi support. The example below will provide a spec for email@example.com using the provided firstname.lastname@example.org and email@example.com packages. This eliminates the need for spack to install additional dependencies.
Install the package using the spack install command.
Since packages are installed from source, some packages may take a while to build. As an example, to install firstname.lastname@example.org using the provided email@example.com and firstname.lastname@example.org packages,
This system has been configured to expose Spack installed packages through lmod. Because of this, you can load them into your environment using module load commands.
Adding Commercial Compilers
Compilers from commercial vendors like Intel and PGI are not open-source and cannot be installed using Spack. However, if you wish to use commercially licensed compilers, we recommend that you install the compilers underneath the /apps directory on the cluster and add an environment module file under /apps/modulefiles.
When installing a module file, we recommend that you make a subdirectory under /apps/modulefiles named after the compiler vendor and name the module file after the version. For example, if I wanted to install PGI compilers version 20.4, the modulefile would be found at /apps/modulefiles/pgi/20.4.
Once your commercial compiler and module file are installed, use the module load command to bring the compiler into your path. Then, use the spack compiler find command with the scope set to "site" to add the compiler to spacks compilers.yaml file. For example
Build your own code
If you'd like to build and install your own code on the cluster, you can easily install packages in your home directory. Alternatively, if you'd like to install your application for use by multiple users, you can follow these recommended best practices
Use spack to install your desired compilers and dependencies with spack install
When building your application, use the spack load command to bring the compilers and dependencies into your path
Install your application under the /apps directory (root privileges are required for this).
Add a module file under the /apps/modulefiles directory so that users can easily bring your application binaries into their path.
Create a Spack Package
We recommend that you consider writing a spack package for your application. This will greatly reduce the barrier to entry for other application users on Fluid-Slurm-GCP and will simplify the installation process on other systems using Spack. If your application's dependencies are already managed by spack, creating a spack package for your application is easy to do.
Create a Package for your Cluster
Create a tagged release of your application and push the tag to your Github, Gitlab, or Bitbucket repository.
In your browser, navigate to your release page and find the .zip or .tar.gz pacakage that contains the tagged release of your application. For example, the package feq-parse has one of its releases hosted on Github at https://github.com/FluidNumerics/feq-parse/releases/tag/v1.0.2.
Use spack create to create a package template for your application. This command will auto-detect your application's build system (ideally, autoconf or cmake) and open a text editor for you to fill out some details for your application. You may need to run this command as root on the cluster, since spack is installed under /apps. For example
Modify the spack package file for your application. See the Spack Packaging Guide for more details.
Test your application install by using spack install
At this point, users on your cluster will be able to install and load your application via Spack.
Contribute your package to the official Spack repository
If you would like to list your package on the official Spack repository, there are a few steps you can take to make the process go as smooth as possible.
Run spack flake8 to verify that your package file passes Spack's formatting requirements. Resolve any issues.
Run git pull to update spack to the most recent version of the develop branch
Navigate to the /apps/spack directory on your cluster and run git pull to update spack to the most recent version of the develop branch.
Run git add and git commit to commit your local changes.
Create a fork of the official spack repository on Github.
Add another remote to your local spack repository on the cluster that points to your fork
Push your changes to this new remote.
Open a pull request with the official spack repository