Using Spack on an "air-gapped" system at the labs

[Jeremiah Wilke]

I want to demo Spack for collaborators on a restricted system - one of them actually at LLNL.

I would guess this is something the Spack team has already thought a lot about. If all my Spack packages point to public github repos/tarball URLs... how do I use Spack when I go to a restricted system that can't touch the web and requires me to do special SCP or file uploads to get files onto the system? 

[Kelly Thompson]

Take a look at spack mirrors --  https://spack.readthedocs.io/en/latest/mirrors.html?highlight=mirror 

I believe you can use "spack fetch" to populate your mirror and then move it to the air-gaped system.

 

-kt
-----
Kelly (KT) Thompson
k...@eklaj.org

On Sun, Apr 7, 2019 at 12:52 PM Jeremiah Wilke <jjwi...@gmail.com> wrote:

I want to demo Spack for collaborators on a restricted system - one of them actually at LLNL.

I would guess this is something the Spack team has already thought a lot about. If all my Spack packages point to public github repos/tarball URLs... how do I use Spack when I go to a restricted system that can't touch the web and requires me to do special SCP or file uploads to get files onto the system? 

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
For more options, visit https://groups.google.com/d/optout.

[Jeremiah Wilke]

Thanks! This looks like exactly what I need. I need to just read the docs, apparently. Spack is TOO well documented, though. Too much to read.

On Sunday, April 7, 2019 at 2:49:50 PM UTC-7, Kelly Thompson wrote:

Take a look at spack mirrors --  https://spack.readthedocs.io/en/latest/mirrors.html?highlight=mirror 

I believe you can use "spack fetch" to populate your mirror and then move it to the air-gaped system.

 

-kt
-----
Kelly (KT) Thompson
k...@eklaj.org

On Sun, Apr 7, 2019 at 12:52 PM Jeremiah Wilke <jjwi...@gmail.com> wrote:

I want to demo Spack for collaborators on a restricted system - one of them actually at LLNL.

I would guess this is something the Spack team has already thought a lot about. If all my Spack packages point to public github repos/tarball URLs... how do I use Spack when I go to a restricted system that can't touch the web and requires me to do special SCP or file uploads to get files onto the system? 

--
You received this message because you are subscribed to the Google Groups "Spack" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sp...@googlegroups.com.

[Jeremiah Wilke]

Okay - adding to the question. We have a fairly involved approval process for moving code to/from the system. In this sense, we would pay a large "startup" cost to move an initial Spack mirror to the system in terms of upload approval. This could take a week or a month. Suppose we want to do weekly updates to the codes. I am only uploading a small "diff" at this point, which should only take a few hours to approve. If I upload an entirely new mirror, the whole thing has to get approved again which will take way too long.

I don't see this in the Spack mirror documentation. In some sense, what I really want is for each Spack mirror to be a Git repo. My upload request would be a git diff, not a new mirror. Is this at all possible with current features? Or would something new like this need to be added?

[Elizabeth A. Fischer]

Is there a spack env fetch to fetch all the tarballs needed for an environment?

To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

[Gamblin, Todd]

You could make the mirror directory a git repo and check in files to record that they’ve already been sent over.

If you do this by just fetching files as KT suggested, you should make var/spack/cache your git repo (var/spack/cache is a special mirror where fetched things get cached). If you do it with `spack mirror create -d ...` it’s whatever directory you passed for `-d`.

---
Sent from Workspace ONE Boxer

To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

[Gamblin, Todd]

See “mirrors”:

https://spack.readthedocs.io/en/latest/mirrors.html

spack mirror create -d <dir> spec [...]

---
Sent from Workspace ONE Boxer

On April 7, 2019 at 11:52:46 AM PDT, Jeremiah Wilke <jjwi...@gmail.com> wrote:

I want to demo Spack for collaborators on a restricted system - one of them actually at LLNL.

I would guess this is something the Spack team has already thought a lot about. If all my Spack packages point to public github repos/tarball URLs... how do I use Spack when I go to a restricted system that can't touch the web and requires me to do special SCP or file uploads to get files onto the system? 

--

[George Hartzell]

Jeremiah Wilke writes:
> [...]

> I don't see this in the Spack mirror documentation. In some sense, what I
> really want is for each Spack mirror to be a Git repo. My upload request
> would be a git diff, not a new mirror. Is this at all possible with current
> features? Or would something new like this need to be added?

At a previous job, my mirror *was* a git repository.
Deleted/added/committed tarballs to it and tagged it to correspond to
"releases" of my application tree. Worked like a charm.

The shell script that built everything included this little bit at the
top to configure Spack to use the mirror (imagine appropriate values
for the SPACK_{ROOT,MIRROR} variables):

```sh
cat << EOYAML > ${SPACK_ROOT}/etc/spack/mirrors.yaml
mirrors:
local: file://${SPACK_MIRROR}
EOYAML
```

g.

[Jeremiah Wilke]

Many thanks for all the answers! After pondering I think I actually want something more fine-grained. I would like to preserve the git histories of the individual packages, if they depend on git repos. Basically replicate github on the other side of the airgap. So a command like:

```

spack mirror-update create

```

This would create git bundles with the new changesets for the all repos. Then I transfer a `mybundle.file` to the system.

```

spack mirror-update apply mybundle.file

```

which would update the git repos.

Am I probably the only person who want this? I don't expect the core Spack team to implement this. I'm basically wondering if we ourselves should try to implement a toolchain outside Spack - or try to integrate with Spack.

[Gamblin, Todd]

I think all that would take would be to tell the git fetch strategy (look at fecth_strategy.py) to:

a) not do a shallow clone

b) include history (.git) when tarring up git repo contents

then you could just push to the high side repos after untarring directories containing git repos.

You said you only want diffs to be transferred, but in not sure that is worth it to only send diffs.  If you really want that, I suppose your tooling could spit out patches, then you could apply them with `git am`. But you’d lose tags and you’d have to manage branches yourself.  Seems like a pain.

Todd

---
Sent from Workspace ONE Boxer

--

You received this message because you are subscribed to the Google Groups "Spack" group.

To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

[Jeremiah Wilke]

I had originally been thinking patches. I actually just posted a question to stackoverflow. People who know git much better than I seem to think `git bundle` provides an easy way to create files holding just the changesets representing a `diff`. I will try a demo version of this and report back : )

To unsubscribe from this group and stop receiving emails from it, send an email to sp...@googlegroups.com.

[Greg Burgreen]

I do this all the time. My approach is:

1. Build my software on an internet open system.

2. Create a tarball of the archive file in var/spack/cache

3. Copy that tarball over to the air-gapped system (including a separate tarball of a recent spack release).

4. Untar the archive tarball into a thirdparty dir in my $SPACK_ROOT

5. Setup a mirrors.yaml

$ cat etc/spack/mirrors.yaml
mirrors:
  local_filesystem: file:///p/work/burgreen/stack-2019.04.05/thirdparty

6. Then, spack away. It retrieves the needed files from my thirdparty dir.

Greg

To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

[George Hartzell]

One more thing to keep in mind: there are a few packages that download
things when you build them (or later). They're not going to find what
they need in the Spack package cache.

I think that the luarocks systems works this way (or did it get
fixed).

I know that NextFlow (java workflow manager) only installs a little
shim and the first time that a user runs it it downloads most of its
bits into `~/.something-or-other` (crazy!).

Not sure what else is sloppy like this.

g.

[Jeremiah Wilke]

Do you mean Spack resources? Or something else?

If spack resources don't end up in the mirror, that could be a deal breaker.

[Gamblin, Todd]

The main version tarballs, resources and patches (if specified by URL) are all mirrored.

I think George means that some packages run stuff in their install() method that ends up fetching.

--
You received this message because you are subscribed to the Google Groups "Spack" group.

To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

[George Hartzell]

'Gamblin, Todd' via Spack writes:
> The main version tarballs, resources and patches (if specified by URL) are all mirrored.
>
> I think George means that some packages run stuff in their install() method that ends up fetching.
>

Yes, I'm describing poorly behaved software that downloads things in
its install() method.

I dug around a bit in the Issues, here's an example:

- Luarocks issue: https://github.com/spack/spack/issues/3909

g.

[Elizabeth A. Fischer]

Are there any specific packages you need to install that won't work on an air-gapped system?  I would focus on fixing them.  IMHO, this is not a Spack problem; but sometimes we can be clever at Spack and make such packages work in spite of themselves.

[George Hartzell]

Elizabeth A. Fischer writes:
> Are there any specific packages you need to install that won't work on an
> air-gapped system? I would focus on fixing them. IMHO, this is not a
> Spack problem; but sometimes we can be clever at Spack and make such
> packages work in spite of themselves.

@elizabeth -- I'm not sure if you're addressing Jeremiah or me. Just
in case: I'm not using Spack for anything professionally right now and
have no problematic packages.

Thanks!

g.