Inconsistent concretization in 'spec' and 'install'

Petr Hanousek

unread,

Dec 13, 2019, 9:07:09 AM12/13/19

to Spack

Hi forum,

does anybody know why I get different results when running 'spack spec' and 'spack install'? The case is:

spack spec -Il boost +mpi+python+numpy %gcc ^python@3.8.0  
Input spec
--------------------------------
 -   boost%gcc+mpi+numpy+python
 -       ^python@3.8.0

Concretized
--------------------------------
 -   nwu6dpo  boost@1.70.0%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +atomic+chrono~clanglibcpp~context~coroutine cxxstd=98 +date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math+mpi+multithreaded+numpy patches=2ab6c72d03dec6a4ae20220a9dfd5c8c572c5294252155b85c6874d97c323199 ~pic+program_options+python+random+regex+s
erialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout visibility=hidden +wave arch=linux-debian9-x86_64                                                                                                                                                                                                             
[+]  yg56no6      ^bzip2@1.0.8%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +shared arch=linux-debian9-x86_64
[+]  mczchqd      ^openmpi@3.1.4%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" ~cuda+cxx_exceptions fabrics=verbs ~java~legacylaunchers~memchecker~pmi schedulers=tm ~sqlite3+thread_multiple+vt arch=linux-debian9-x86_64
[+]  squfgvb          ^hwloc@1.11.11%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" ~cairo~cuda~gl+libxml2~nvml+pci+shared arch=linux-debian9-x86_64
[+]  4ot2qhu              ^libiconv@1.16%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  y25qs2g              ^libpciaccess@0.13.5%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  o4dh2ls                  ^libtool@2.4.6%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  nlttomf                  ^pkgconf@1.6.3%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  6jan4ho                  ^util-macros@1.19.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  lr4lkvu              ^libxml2@2.9.9%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" ~python arch=linux-debian9-x86_64
[+]  4shwjcj                  ^xz@5.2.4%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  tj6bxtz                  ^zlib@1.2.11%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +optimize+pic+shared arch=linux-debian9-x86_64
[+]  lnr5so7              ^numactl@2.0.12%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  n4pcbri                  ^autoconf@2.69%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  ywsowxy                  ^automake@1.16.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  ag2vlez                  ^m4@1.4.18%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-debian9-x86_64
 -   ur5oqwh      ^py-numpy@1.17.3%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +blas+lapack arch=linux-debian9-x86_64
 -   jbiuyqf          ^intel-mkl@2019.4.243%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" ~ilp64+shared threads=none arch=linux-debian9-x86_64
[+]  yaeapw6          ^py-setuptools@41.4.0%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  rab6tdg              ^python@3.8.0%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +bz2+ctypes+dbm+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4~uuid+zlib arch=linux-debian9-x86_64
[+]  467n6q4                  ^expat@2.2.9%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +libbsd arch=linux-debian9-x86_64
[+]  2cpo2y5                      ^libbsd@0.9.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  uie3d7j                  ^gdbm@1.18.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  wrpnkym                      ^readline@8.0%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  xvp6spa                          ^ncurses@6.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +symlinks~termlib arch=linux-debian9-x86_64
[+]  f7npdp3                  ^gettext@0.20.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-debian9-x86_64
[+]  yruiqtc                      ^tar@1.32%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  cdmaxmj                  ^libffi@3.2.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC"  arch=linux-debian9-x86_64
[+]  vu2s35g                  ^openssl@1.1.1d%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +systemcerts arch=linux-debian9-x86_64
[+]  gjxqb2l                      ^perl@5.30.0%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" +cpanm+shared+threads arch=linux-debian9-x86_64
[+]  dn52zhd                  ^sqlite@3.30.1%gcc@9.2.0 cflags="-fPIC" cxxflags="-fPIC" fflags="-fPIC" ~column_metadata+fts~functions~rtree arch=linux-debian9-x86_64

And then when running install I see Python is installing again:

spack install boost +mpi+python+numpy %gcc ^python@3.8.0         
==> bzip2@1.0.8 : externally installed in /usr
==> bzip2@1.0.8 : already registered in DB
==> libiconv is already installed in /cvmfs/software.metacentrum.cz/software/libiconv/linux-debian9-x86_64/1.16-gcc-4ot2q
==> libtool@2.4.6 : externally installed in /usr
==> libtool@2.4.6 : already registered in DB
==> pkgconf@1.6.3 : externally installed in /cvmfs/software.metacentrum.cz/software_basics/pkgconf/1.6.3
==> pkgconf@1.6.3 : already registered in DB
==> util-macros is already installed in /cvmfs/software.metacentrum.cz/software/util-macros/linux-debian9-x86_64/1.19.1-gcc-6jan4
==> libpciaccess is already installed in /cvmfs/software.metacentrum.cz/software/libpciaccess/linux-debian9-x86_64/0.13.5-gcc-y25qs
==> xz is already installed in /cvmfs/software.metacentrum.cz/software/xz/linux-debian9-x86_64/5.2.4-gcc-4shwj
==> zlib is already installed in /cvmfs/software.metacentrum.cz/software/zlib/linux-debian9-x86_64/1.2.11-gcc-tj6bx
==> libxml2 is already installed in /cvmfs/software.metacentrum.cz/software/libxml2/linux-debian9-x86_64/2.9.9-gcc-lr4lk
==> autoconf@2.69 : externally installed in /usr
==> autoconf@2.69 : already registered in DB
==> automake@1.16.1 : externally installed in /usr
==> automake@1.16.1 : already registered in DB
==> m4@1.4.18 : externally installed in /usr
==> m4@1.4.18 : already registered in DB
==> numactl is already installed in /cvmfs/software.metacentrum.cz/software/numactl/linux-debian9-x86_64/2.0.12-gcc-lnr5s
==> hwloc is already installed in /cvmfs/software.metacentrum.cz/software/hwloc/linux-debian9-x86_64/1.11.11-gcc-squfg
==> openmpi is already installed in /cvmfs/software.metacentrum.cz/software/openmpi/linux-debian9-x86_64/3.1.4-gcc-mczch
==> intel-mkl@2019.4.243 : externally installed in /cvmfs/software.metacentrum.cz/software_basics/intel-mkl/2019.4.243
==> intel-mkl@2019.4.243 : already registered in DB
==> libbsd is already installed in /cvmfs/software.metacentrum.cz/software/libbsd/linux-debian9-x86_64/0.9.1-gcc-2cpo2
==> expat is already installed in /cvmfs/software.metacentrum.cz/software/expat/linux-debian9-x86_64/2.2.9-gcc-467n6
==> ncurses is already installed in /cvmfs/software.metacentrum.cz/software/ncurses/linux-debian9-x86_64/6.1-gcc-xvp6s
==> readline is already installed in /cvmfs/software.metacentrum.cz/software/readline/linux-debian9-x86_64/8.0-gcc-wrpnk
==> gdbm is already installed in /cvmfs/software.metacentrum.cz/software/gdbm/linux-debian9-x86_64/1.18.1-gcc-uie3d
==> tar@1.32 : externally installed in /usr
==> tar@1.32 : already registered in DB
==> gettext is already installed in /cvmfs/software.metacentrum.cz/software/gettext/linux-debian9-x86_64/0.20.1-gcc-2jdbj
==> libffi is already installed in /cvmfs/software.metacentrum.cz/software/libffi/linux-debian9-x86_64/3.2.1-gcc-cdmax
==> perl@5.30.0 : externally installed in /usr
==> perl@5.30.0 : already registered in DB
==> openssl is already installed in /cvmfs/software.metacentrum.cz/software/openssl/linux-debian9-x86_64/1.1.1d-gcc-vu2s3
==> sqlite is already installed in /cvmfs/software.metacentrum.cz/software/sqlite/linux-debian9-x86_64/3.30.1-gcc-dn52z
==> Installing python
==> Searching for binary cache of python
==> Warning: No Spack mirrors are currently configured
==> No binary for python found: installing from source
==> Using cached archive: /scratch/spack/source_cache/_source-cache/archive/f1/f1069ad3cae8e7ec467aa98a6565a62a48ef196cb8f1455a245a08db5e1792df.tgz
==> Staging archive: /ramdisk/hanousek/spack-stage-python-3.8.0-6okiz6uqbtrrsdhxyb3srq6eytw2mnq3/Python-3.8.0.tgz
==> Created stage in /ramdisk/hanousek/spack-stage-python-3.8.0-6okiz6uqbtrrsdhxyb3srq6eytw2mnq3
==> Ran patch() for python
==> Building python [AutotoolsPackage]
....

Why has Python package suddenly the different hash? Running the last stable Spack version.

Thank you, Petr

Gamblin, Todd

unread,

Dec 13, 2019, 1:02:18 PM12/13/19

to Petr Hanousek, Spack

Petr:

Are you running `spack spec` in an environment with different defaults? It looks like it as you’re getting all the -fPIC flags and the arch is x86_64. Are you sure you’re in the same environment as when you run `spack install`? Note that if you’re in a directory with a spack.yaml, you’ll pick up config from that spack.yaml as well.

Even if it’s not related, we should probably add something to `spack spec` and spack install saying what environment we’re in.

-Todd

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/4f792483-4927-4a66-85cd-d236cf3fa83a%40googlegroups.com.

Petr Hanousek

unread,

Dec 13, 2019, 1:18:27 PM12/13/19

to Spack

Hi Todd,

this both was run without any environment.

Petr

Dne pátek 13. prosince 2019 19:02:18 UTC+1 Todd Gamblin napsal(a):

Petr Hanousek

unread,

Dec 13, 2019, 1:20:17 PM12/13/19

to Spack

Sorry for incomplete answer, the -fPIC directives are defined globally in $SPACK_ROOT/etc/spack/compilers.yaml

Petr

Dne pátek 13. prosince 2019 19:18:27 UTC+1 Petr Hanousek napsal(a):

Petr Hanousek

unread,

Jan 6, 2020, 4:29:51 AM1/6/20

to Spack

I've seen #14390. Maybe the mentioned bug can cause also this problem?

BTW now I've tried 'git pull' on 'releases/v0.13' and the same commands as reported without any environment. Suddenly Spack wants (in 'spack spec') install also the exactly the same package 'py-setuptools' as it was satisfied in my previous question. And still the python package.

BTW2 as you can see, in 'spack spec' part is missing 'intel-mkl' package but during 'spack install' is correctly recognized the already installed one.