Installing a list of packages from a file

38 views
Skip to first unread message

Rob G

unread,
May 14, 2019, 11:38:15 AM5/14/19
to Spack

I swear that when I was reading the spack documents, it mentioned somewhere that "You can execute several separate spack install statements for each package you'd like to install, or you can define the packages to install in a file and use that", but I CANNOT find that reference now.

I know that the command "spack install --file <something.yaml>" exists, but I cannot find any reference for what shape this file takes internally.

Basically, I want to be able to define the 30 packages I want installed on the system so that I can go to any system, say "spack install" and it will install those same packages.  If that's "spack install -f", then what goes in the .yaml file for those 30 packages?

Thanks.


Rob

Gamblin, Todd

unread,
May 14, 2019, 11:49:42 AM5/14/19
to Rob G, Spack

Hey Rob:

 

Take a look at Spack environments:

 

                https://spack.readthedocs.io/en/latest/tutorial_environments.html

 

You can make a spack.yaml file like this:

 

spack:

    specs:

        - hdf5

        - zlib

 

Add your 3- packages to the list, cd to the directory containing spack.yaml, and type `spack install`.  A `spack.lock` file will be spit out that you can use to create the environment exactly the same way elsewhere.

 

-Todd

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/6cb6ed70-787d-4555-ad08-732605ad5e60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

v

unread,
May 14, 2019, 11:49:54 AM5/14/19
to Rob G, Spack

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/6cb6ed70-787d-4555-ad08-732605ad5e60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Vanessa Villamia Sochat
Stanford University '16

Gamblin, Todd

unread,
May 14, 2019, 11:58:25 AM5/14/19
to v, Rob G, Spack

packages.yaml configuration tells you *how* to concretize packages (and which external packages to use) but not *what* to install.  You can add a packages: section to your spack.yaml file to include those types of configuration settings with your environment.

 

I guess I should enumerate a few other things for those interested in the finer details:

  • `spack install -f` *used to* take a file that had a list of specs, one per line (same thing you’d pass to spack install).  We’ve replaced this with the spack.yaml format from environments because:
    • spack.yaml can contain both specs and concretization preferences as you’d find in spack.yaml, so it’s more flexible
    • running spack install on an environment also spits out spack.lock, which you can use to exactly reproduce a prior install (the previous file format didn’t allow anything like this)
  • `spack install -f` *now* accepts the Spack’s spec YAML format, which is what you get when you run `spack spec -y <spec>`.  This is really intended for distributed builds and build pipelines to use.  It’s intended to run a *single* install with full information from a prior spack spec invocation.

 

Sorry for the confusion!

 

-Todd

Rob G

unread,
May 14, 2019, 12:04:45 PM5/14/19
to Spack
 I'm guessing that's what I want.  My goal is to easily and reliably re-create the same list of available apps, for any newly built machine.  So if I can just say "spack install" and the spack.yaml file will tell it what to install, then that's what I'm looking for.  Thanks Todd.

Rob


On Tuesday, May 14, 2019 at 11:49:42 AM UTC-4, Todd Gamblin wrote:

Hey Rob:

 

Take a look at Spack environments:

 

                https://spack.readthedocs.io/en/latest/tutorial_environments.html

 

You can make a spack.yaml file like this:

 

spack:

    specs:

        - hdf5

        - zlib

 

Add your 3- packages to the list, cd to the directory containing spack.yaml, and type `spack install`.  A `spack.lock` file will be spit out that you can use to create the environment exactly the same way elsewhere.

 

-Todd

 

 

From: <sp...@googlegroups.com> on behalf of Rob G <rob....@gmail.com>
Date: Tuesday, May 14, 2019 at 5:38 PM
To: Spack <sp...@googlegroups.com>
Subject: [spack] Installing a list of packages from a file

 

 

I swear that when I was reading the spack documents, it mentioned somewhere that "You can execute several separate spack install statements for each package you'd like to install, or you can define the packages to install in a file and use that", but I CANNOT find that reference now.

 

I know that the command "spack install --file <something.yaml>" exists, but I cannot find any reference for what shape this file takes internally.

 

Basically, I want to be able to define the 30 packages I want installed on the system so that I can go to any system, say "spack install" and it will install those same packages.  If that's "spack install -f", then what goes in the .yaml file for those 30 packages?

 

Thanks.

 

 

Rob

 

--
You received this message because you are subscribed to the Google Groups "Spack" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sp...@googlegroups.com.

Gamblin, Todd

unread,
May 14, 2019, 12:15:34 PM5/14/19
to Rob G, Spack
Yep, and tha spack.yaml file is designed to be versioned in a git repo, so you should also be able to version custom repositories, etc. that you might want.

You may also be interested in some extensions we’re adding to the format, mostly to make large deployments at HPC centers easier:


Todd



---
Sent from Workspace ONE Boxer

To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.

Rob G

unread,
May 14, 2019, 12:19:15 PM5/14/19
to Spack
I am indeed interested in that, as I'm evaluating spack for our HPC cluster.  Thank you!

Rob


On Tuesday, May 14, 2019 at 12:15:34 PM UTC-4, Todd Gamblin wrote:
Yep, and tha spack.yaml file is designed to be versioned in a git repo, so you should also be able to version custom repositories, etc. that you might want.

You may also be interested in some extensions we’re adding to the format, mostly to make large deployments at HPC centers easier:


Todd



---
Sent from Workspace ONE Boxer

Elizabeth A. Fischer

unread,
May 14, 2019, 12:22:23 PM5/14/19
to Rob G, Spack
On Tue, May 14, 2019 at 12:19 PM Rob G <rob.g...@gmail.com> wrote:
I am indeed interested in that, as I'm evaluating spack for our HPC cluster.  Thank you!

Rob


On Tuesday, May 14, 2019 at 12:15:34 PM UTC-4, Todd Gamblin wrote:
Yep, and tha spack.yaml file is designed to be versioned in a git repo, so you should also be able to version custom repositories, etc. that you might want.

See here for an example of a Spack Environment versioned in a Git repo, along with other changes needed for our installation.  (Mostly package changes that we need, that haven't yet been merged back into main Spack).


 -- Elizabeth

Rob G

unread,
May 14, 2019, 2:12:15 PM5/14/19
to elizabet...@columbia.edu, Spack
Thanks, I'll check it out. 
Reply all
Reply to author
Forward
0 new messages