hashdist installation without internet access

[Mario Schreiber]

Hello,
I am trying to install Fenics in a managed environment, where I cannot download any packages by http, ftp or git (no internet access), and I cannot setup NAT, because I have no superuser permissions. I am using the hashdist installation method of Fenics, but it fails at the point, when the first dependencies should be downloaded.

Is it possible to first perform a 'dry' hashdist run on another system (with internet access), which downloads all packages but doesn't build anything in this step? Secondly, I would transfer all downloaded packages to the managed environment. Finally, I would run hashdist there, which works with the uploaded packages, downloads nothing and builds everything. How would I tell hashdist either not to build anything (1st step) or not to download anything (last step)?

Or can you recommend other solutions, how to perform a hashdist installation from local sources only, and how to automatically create the set of sources from a hashstack  configuration?

Thank you
Marion

[Ondřej Čertík]

Hi Mario,

On Sun, Dec 18, 2016 at 2:53 PM, Mario Schreiber
<schreibe...@gmail.com> wrote:
> Hello,
> I am trying to install Fenics in a managed environment, where I cannot
> download any packages by http, ftp or git (no internet access), and I cannot
> setup NAT, because I have no superuser permissions. I am using the hashdist
> installation method of Fenics, but it fails at the point, when the first
> dependencies should be downloaded.
>
> Is it possible to first perform a 'dry' hashdist run on another system (with
> internet access), which downloads all packages but doesn't build anything in
> this step? Secondly, I would transfer all downloaded packages to the managed

I usually build it on another machine with an internet access, then
copy the src directory (with all the sources) onto a machine with no
internet connection.

I created an issue for this:

https://github.com/hashdist/hashdist/issues/370

> environment. Finally, I would run hashdist there, which works with the
> uploaded packages, downloads nothing and builds everything. How would I tell
> hashdist either not to build anything (1st step) or not to download anything
> (last step)?

It will not download anything if it has access to it in the sources directory.

>
> Or can you recommend other solutions, how to perform a hashdist installation
> from local sources only, and how to automatically create the set of sources
> from a hashstack configuration?

Let me know if the above works for you.

Ondrej


>
> Thank you
> Marion
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "hashdist" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to hashdist+u...@googlegroups.com.
> To post to this group, send email to hash...@googlegroups.com.
> Visit this group at https://groups.google.com/group/hashdist.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hashdist/0fef4b64-4ca4-48e8-b726-d659f549711d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Mario Schreiber]

Hello Ondrej,
it is good to know, that there is a possibility to upload packages and install from those.
I have some problems with the details, though: When I build some hashdist profile on a system with internet access, it seems like the *tar.gz packages are deleted during the process. Where exactly do I find the sources? Somewhere below $HOME/.hashdist/? I think it will be problematic if I archive those directories, in which the packages have been configured and built, firstly because they are not configured for the system where I want to use them later, secondly because the binaries will waste space.

Then it is not very practical to build a rather large profile like Fenics for the only purpose of downloading the packages. If there is no solution in hashdist, I'll probably switch to spack or maybe parse the *.yaml files to do the downloads independent from hashdist.

Thanks for your reply
Mario

[Aron Ahmadia]

By default the source tarballs will live in the ~/.hashdist/src directory. That's the only one you'd need to copy.

A

To view this discussion on the web visit https://groups.google.com/d/msgid/hashdist/8d99178c-4d96-4925-8bd0-de696c91927f%40googlegroups.com.

[Mario Schreiber]

For future reference, all this is easily solved with spack manager, so it makes sense to switch.

Anyway, thanks for your assistance and for filing an issue.

Mario

[Mario Schreiber]

Am Montag, 19. Dezember 2016 18:32:47 UTC+1 schrieb ondrej.certik:

I usually build it on another machine with an internet access, then
copy the src directory (with all the sources) onto a machine with no
internet connection.

This does not work, either. During the fenics installation, a few packages are installed from the copied src directory. Then the installation hangs at the ncurses package, which hashdist tries to download although it is available in ~/.hashdist/src/git/ncurses.

[Ondřej Čertík]

If you build with "-v", it should print what it's doing. Is it
downloading the ncruses package, or is it something during its build?
Either way, that sounds like a bug.

Ondrej