What's the approved way to create build caches for everything I build?

[george hartzell]

I currently have a script that does a bunch of `spack install spec` commands to build the things that I use.  I'm exploring moving towards environments but am still learning about them.

I'd like to create a build cache entry for everything I build.

My first take was to replace my calls to `spack install spec` with a call to this function:

    install () {

        spack install $1

        spack buildcache create -d ${BUILDCACHE} -a -u -f $1

    }

It's still running, but seems to be working.  It seems suboptimal in that it recreates build caches for dependencies (hence the `-f`).

I also played around with just creating build caches for everything in an installation with this:

    for s in $(spack find --no-groups -L | cut -f 1 -d ' ' ); do

      spack buildcache create -d build_cache -a -u --only package "/$s";

    done

It also seems to have done what I need but doesn't feel very elegant.

So:

- any improvements on either of those approaches?

- is there anything from the environment world that'll make this easier?

Thanks!

g.

[Quellyn Snead]

Hey George, I'm in the same boat as you are: I want to make a large build_cache for our common dependencies, to save our developers from reinventing the wheel. Did you ever find a better way to do this?

Thanks!

Q

[George Hartzell]

I currently run a series of `spack install this`, `spack install that`
and the have this bit at the bottom of the script (I suspect I cribbed
it from somewhere but might have cons'ed it up myself):

```
# Grab the hashes of all of the installed packages and use them to
# populate the buildcache.

for s in $(spack find --no-groups -L | cut -f 1 -d ' ' ); do

spack buildcache create -d ${BUILDCACHE_MIRROR} -a -u --only package "/$s"
done
```

g.

> --
> You received this message because you are subscribed to the Google Groups "Spack" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spack/6450dd67-d6e0-467c-9265-80dda9050ca8n%40googlegroups.com.

[Quellyn Snead]

Ah, okay. We're using Spack environments here and I had been just adding
things to the buildcache piecemeal:

spack env activate <path-to-spack.yaml>
spack buildcache create -a -m <my-mirror-name> -r -u
<installed-package-name>

I kept thinking there must be a way to just pass the spack.yaml file to
"spack buildcache create", but could not find any example of that. I
think I'll adopt your little trick of just reaping the hashes (or
perhaps names) for just the packages installed in that environment.

Thanks!
Q

--
Quellyn L Snead
XCP-1, Lagrangian Codes
Los Alamos National Laboratory
Office: 505-667-4185

[George Hartzell]

Quellyn Snead writes:
> Ah, okay. We're using Spack environments here and I had been just adding
> things to the buildcache piecemeal:
>
> spack env activate <path-to-spack.yaml>
> spack buildcache create -a -m <my-mirror-name> -r -u
> <installed-package-name>
>
> I kept thinking there must be a way to just pass the spack.yaml file to
> "spack buildcache create", but could not find any example of that.

> [...]

A nice solution might be able to hold buildcache building in the same
way modulefiles are built, so that whenever you build a package you
also build the buildcache.

g.

[Quellyn Snead]

That would be ideal, yes.  :-)