[Cbr 954 Vs Ninja 636 Specs
Facunda Ganesh
This section of the tutorial introduces Spack Environments, which allow youto work with independent groups of packages separately, in a reproducible way.In some ways, Spack environments are similar to virtual environments in othersystems (e.g., Python venv),but they are based around file formats (spack.yaml and spack.lock) that canbe shared easily and re-used by others across systems.
Administering properly configured software involving lots of packagesand/or varying configuration requirements (e.g., different implementationsof mpi) for multiple projects and efforts can be overwhelming. Spackenvironments allow you to readily:
This tutorial introduces the basics of creating and using environments,then explains how to expand, configure, and build software in them.We will start with the command line interface, then cover editing keyenvironment file directly. We will describe the difference betweenSpack-managed and independent environments, then finish with a sectionon reproducible builds.
An environment is like a virtualized Spack instance that you canuse to aggregate package installations for a project or other purpose.It has an associated view, which is a single prefix where all packagesfrom the environment are linked.
If you create environments incrementally, Spack ensures that already installedroots are not re-concretized. So, adding specs to an environment at a later pointin time will not cause existing packages to rebuild.
Environments provide a convenient way for using installed packages.Running spack env activate gives you everything in the environmenton your PATH. Otherwise, you would need to use spack loador module loadfor each package in order to set up the environment for the package (andits dependencies).
When you install packages into an environment, they are, by default,linked into a single prefix, or view. Activating the environmentwith spack env activate results in subdirectories from the viewbeing added to PATH, MANPATH, CMAKE_PREFIX_PATH,and other environment variables. This makes the environment easier to use.
concretizer:unify:true, the default, means that they are concretizedtogether, so that there is only one version of each package in theenvironment. Other options for unify are false and when_possible.false means that the specs are concretized independently, so thatthere may be multiple versions of the same package in the environment.when_possible lies between those options. In this case, Spack will unifyas many packages in the environment, but will not fail if it cannot unifyall of them.
You may need to re-install packages in the environment after makingsignificant changes to the configuration, such as changing virtualproviders. This can be accomplished by forcing Spack to re-concretizethe environment and re-install the specs.
For example, the packages installed in our myproject environmentare now out of sync with our new configuration since we alreadyinstalled part of the environment with openmpi. Suppose we wantto install everything in myproject with mpich.
Re-concretization is sometimes also necessary when creating anenvironment incrementally with unification enabled. Spack makessure that already concretized specs in the environment are notmodified when adding something new.
Adding and installing specs one by one leads to greedyconcretization. When you first install python in an environment,Spack will pick a recent version for it. If you then add py-numpy,it may be in conflict with the python version already installed,and fail to concretize:
As mentioned before, activating the environment sets a number ofenvironment variables. That includes variables like PATH,MANPATH, and CMAKE_PREFIX_PATH, which allows you toeasily find package executables and libraries installed in the environment.
There are two key files tracking the contents of environments:spack.yaml and spack.lock. The spack.yaml file holdsthe environment configuration that we previously edited throughspack config edit. The spack.lock file is automaticallygenerated during concretization.
Environments are either Spack-managed or independent. Both typesof environments are defined by their environment files. So farwe have only created managed environments. This section describestheir differences.
Independent environments can be created in one of two ways. First,the Spack environment file(s) can be placed in any directory(other than var/spack/environments). Alternatively, you canuse spack env create -d to specify the directory() in which the files should reside. Independentenvironments are not named.
Notice that myproject is a subdirectory of var/spack/environmentswithin the Spack installation making it a managed environment.Consequently, it can be referenced by name. It will also show upwhen running spack env list:
We can also see from the listing above that the current environment directorycontains both of the environment files: spack.yaml and spack.lock.This is because spack.lock was generated when we concretizedthe environment.
Environments do not have to be created in or managed by a Spackinstance. Rather, their environment files can be placed in anydirectory. This feature can be quite helpful for use cases suchas environment-based software releases and CI/CD.
Notice that the command shows Spack created the environment, updatedthe view, and printed the command needed to activate it. As wecan see in the activation command, since the environment is independent,it must be referenced by its directory path.
Notice that Spack created a spack.yaml file in the code directory.Also note that the configuration file has an empty spec list (i.e.,[]). That list is intended to contain only the root specs ofthe environment.
Spack supports tweaking an environment even after the initial specsare installed. You are free to add and remove specs just as you wouldoutside of the environment using the command line interface as before.
Our focus so far has been on the abstract environment configurationrepresented by the spack.yaml file. Once that file is concretized,Spack generates a corresponding spack.lock file representingthe full concretized state of the environment.
You can recreate an environment by passing either of the environmentfiles to spack env create. The file you choose depends on whetheryou want to approximate the build using the abstract specs or an exactbuild based on the concrete specs.
An approximate build is created using the spack.yaml file. Thisapproach is relevant when we want to build the same specs on a newplatform, for example. It allows you to reproduce the environmentby preserving the abstract requirements in the file. However, thesoftware may actually build differently in part because the concretizermay choose different dependencies.
Since we created the environment from our spack.lock file,not only do we get the same root specs, all of the packages areinstalled in the environment as we can see from callingspack find after activating the environment:
795a8134c1