Installing SPACK on a machine that has no internet access

[Dash]

Hi all,

I'm trying to install Spack on a supercomputer that has no internet access. I tried to create a mirror with all the necessary packages and dependencies need for initial setup (re2c, clingo) but when i do spack spec zlib, spack still tries to install clingo from github and times out.

spack spec zlib

Input spec

--------------------------------

zlib

Concretized

--------------------------------

==> Bootstrapping clingo from sources

[+] /usr (external bison-3.0.4-74yyvywdqdaszgealacc6hn7e6zsv3jb)

[+] /usr (external cmake-3.18.2-5xep2nwnqffvflzhqby4y2zvcm2ybqao)

[+] /usr (external python-3.6-pbfezbjma3snkvc6s7flysgd5yzeguru)

[+] /home/uchu/prathamesh.dash/.spack/bootstrap/store/linux-centos8-x86_64/gcc-8.4.1/re2c-2.2-p5fqievdqfxl3jwocvkwla4m27fdm2pq

==> Installing clingo-bootstrap-spack-vcukpckb6ggkpe734fubvh3ppq5c4rbe

==> No binary for clingo-bootstrap-spack-vcukpckb6ggkpe734fubvh3ppq5c4rbe found: installing from source

fatal: unable to access 'https://github.com/potassco/clingo.git/': Failed to connect to github.com port 443: Connection timed out

==> Error: FetchError: All fetchers failed for spack-stage-clingo-bootstrap-spack-vcukpckb6ggkpe734fubvh3ppq5c4rbe

/sc/home/prathamesh.dash/spack/lib/spack/spack/package.py:1395, in do_fetch:

       1392

       1393        self.stage.create()

       1394        err_msg = None if not self.manual_download else self.download_instr

  >>   1395        start_time = time.time()

       1396        self.stage.fetch(mirror_only, err_msg=err_msg)

       1397        self._fetch_time = time.time() - start_time

       1398

==> Error: cannot bootstrap the "clingo" Python module from spec "clingo-bootstrap@spack+python %gcc target=x86_64" due to the following failures:

    'spack-install' raised InstallError: Terminating after first install failure: FetchError: All fetchers failed for spack-stage-clingo-bootstrap-spack-vcukpckb6ggkpe734fubvh3ppq5c4rbe

    Please run `spack -d spec zlib` for more verbose error messages




I've been stuck trying to solve this for the last two days but can't figure it out

Thanks,
Dash

[Groner, Rob]

I don't think I've tried to mirror a git repo, so I'm not sure how spack does it. When you said you tried to create a mirror...how did you do that?  Did you use the spack mirror command, or did you manually create the directory structure how you thought it should go?

The only two things I can suggest:

1. use "spack mirror list" and make sure your mirror is there and first in the list.
2. On a computer that does have internet access, tell spack "spack mirror create -d <path to a mirror directory> zlib@<version>".  It should then create the mirror you need.  copy that mirror to your offline computer mirror.

Rob

---

From: sp...@googlegroups.com <sp...@googlegroups.com> on behalf of Dash <pratha...@gmail.com>
Sent: Monday, January 31, 2022 10:36 AM
To: Spack <sp...@googlegroups.com>
Subject: [spack] Installing SPACK on a machine that has no internet access

 

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/ba9689d3-6e8b-4d1a-94c0-a9596c698286n%40googlegroups.com.

[Prathamesh Dash]

Hi Rob, Greg,

Rob, I used spack mirror command but still spack is trying to download clingo from the github rep  instead of accessing it locally. I made sure that spack mirror list shows the mirror and also changed the precedence in .spack/mirrors.yaml. 

Greg, I'm not able to figure out which file in your github helps my problem.

Thanks,
Dash

[Greg Burgreen]

Dash,

This is not a single file solution. It is a process. Follow the directions in https://github.com/burgreen/stack-proteus, and then modify for your specific application.

Greg

To view this discussion on the web visit https://groups.google.com/d/msgid/spack/CAC-hDFwQH3RpifZkprNfCxP%3DmZJCLYEHvnPw8Th2HCTWp9XjPQ%40mail.gmail.com.

[Prathamesh Dash]

Hi Angel,

I'm not great at Networking so I'm not sure how to do what you're suggesting. The admin of the computer would never let me set up any kind of proxy server though.

Thanks, 

Dash

On Tue, 1 Feb 2022, 02:08 Angel de Vicente, <angel.de...@iac.es> wrote:

Hello,

Dash <pratha...@gmail.com> writes:

> I'm trying to install Spack on a supercomputer that has no internet
> access. I tried to create a mirror with all the necessary packages and
> dependencies need for initial setup (re2c, clingo) but when i do spack
> spec zlib, spack still tries to install clingo from github and times
> out.

when you say the supercomputer has no internet access I assume you mean
outside connections from the supercomputer are blocked?

If that is the case, and depending on what you want to achieve, perhaps
it is just easier to create a reverse tunnel when you connect to the
supercomputer, and via a proxy connect to the internet? (that is how I
do it in a couple of supercomputers I have access to, which don't allow
outside connections, in order use Spack, and other stuff, without
special setups).

Cheers,
--
Ángel de Vicente

Tel.: +34 922 605 747
Web.: http://research.iac.es/proyecto/polmag/
---------------------------------------------------------------------------------------------
AVISO LEGAL: Este mensaje puede contener información confidencial y/o privilegiada. Si usted no es el destinatario final del mismo o lo ha recibido por error, por favor notifíquelo al remitente inmediatamente. Cualquier uso no autorizadas del contenido de este mensaje está estrictamente prohibida. Más información en: https://www.iac.es/es/responsabilidad-legal
DISCLAIMER: This message may contain confidential and / or privileged information. If you are not the final recipient or have received it in error, please notify the sender immediately. Any unauthorized use of the content of this message is strictly prohibited. More information:  https://www.iac.es/en/disclaimer