Qu re overriding recipes in repos

[Thomas M. Payerle]

I was experimenting with Spack repositories, and I am seeing some behavior which does not match my expectations.  As I am new to repos, I wanted to run my experiences by others to determine if Spack is behaving oddly or if my expectations are awry.

For consistency in builds across clusters, platforms, etc., I am trying to have a "frozen" clone of spack with which to build packages from the "builtin" repo, and when issues with the build of a specific package arise, put the new, fixed recipes in a "fixed" repo (either backporting a newer recipe from spack master, or fixing/adding the recipe (and giving a pull request back to spack master)). 

I am using

> spack debug report

* **Spack:** 0.16.1-2429-f5e6c32495
* **Python:** 3.6.8
* **Platform:** linux-rhel8-x86_64
* **Concretizer:** original

with repos

> spack repos list

hacks.umd.edu     /software/spack/local-repos/umd-hacks-spack-repo
fixed.umd.edu     /software/spack/local-repos/spack-fixed-spack-repo
builtin           /software/spack/git.2021.04.28/var/spack/repos/builtin
extras.umd.edu    /software/spack/local-repos/umd-extras-spack-repo

"hacks.umd.edu" is intended for hacked recipes not suitable for sending back to spack,

"fixed.umd.edu" is the "fixed" repo discussed above, and "extras.umd.edu" for local stuff.  "hacks" and "fixed" should be disjoint, but will override packages in "builtin".  "extras" should

be disjoint from all of the others.  "hacks" and "fixed" come before "builtin", and my understanding was that the recipes for packages in these repos should be found and used before any recipes for packages in builtin.

I have made a modification to the libtirpc package, and placed in "fixed.umd.edu" repo.  I now

have

> spack find -lvfN libtirpc
==> 2 installed packages
-- linux-rhel8-x86_64 / g...@8.4.0 -------------------------------
45klwwn builtin....@1.2.6%gcc  

arpdkiz fixed.umd.e...@1.2.6%gcc

But when I do

> >  spack --env gcc840 spec -lIN hdf
Input spec
--------------------------------
 -   .hdf

Concretized
--------------------------------
 -   5onshc4  built...@4.2.14%g...@8.4.0+external-xdr+fortran~java~netcdf+pic~shared+szip arch=linux-rhel8-x86_64
[+]  uwlvr4f      ^builti...@3.7.4%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  vzpvxcr          ^builtin.diffutils@3.7%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  hwsmrzj              ^builtin....@1.16%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  ln4uzp4          ^builtin....@1.47.16%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  ozedwyl              ^builtin...@0.20.1%g...@8.4.0+bzip2+curses+git+libunistring+libxml2+tar+xz arch=linux-rhel8-x86_64
[+]  s6eoshz                  ^fixed.umd...@1.0.8%g...@8.4.0~debug+pic+shared arch=linux-rhel8-x86_64
[+]  qcg4tps                  ^builtin.li...@0.9.10%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  l3oo5q2                  ^builtin...@2.9.9%g...@8.4.0~python arch=linux-rhel8-x86_64
[+]  2ykulq6                      ^builtin.p...@0.29.2%g...@8.4.0+internal_glib arch=linux-rhel8-x86_64
[+]  ovxv3fh                      ^built...@5.2.5%g...@8.4.0+pic arch=linux-rhel8-x86_64
[+]  wwi45nr                      ^builti...@1.2.11%g...@8.4.0+optimize+pic+shared arch=linux-rhel8-x86_64
[+]  f5r24q7                  ^builtin.ncurses@6.2%g...@8.4.0+symlinks+termlib abi=5 arch=linux-rhel8-x86_64
[+]  7wit6jt                  ^built...@1.34%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  czgyu5r              ^builti...@5.30.1%g...@8.4.0+cpanm+shared+threads arch=linux-rhel8-x86_64
[+]  5xksxkj                  ^builtin.b...@18.1.40%g...@8.4.0+cxx+docs+stl arch=linux-rhel8-x86_64
[+]  dw52ox6                  ^builti...@1.19%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  ppc6pwi                      ^builtin.readline@8.1%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  t5gav5z          ^built...@1.4.18%g...@8.4.0+sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 arch=linux-rhel8-x86_64
[+]  bs2pcxw              ^builtin.l...@2.13%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  fh3wgkk      ^builti...@2.6.4%g...@8.4.0+lex patches=09c22e5c6fef327d3e48eb23f0d610dcd3a35ab9207f12e0f875701c677978d3 arch=linux-rhel8-x86_64
[+]  ux2zeib          ^builtin....@2.69%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  z2p6knn          ^builtin....@1.16.1%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  z2ibdqd          ^builtin....@4.8.0%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  hxg76fv          ^builtin...@2.4.6%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  lgyflwb      ^builtin.libjpeg@9d%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  3tn6tou      ^builtin...@2.1.1%g...@8.4.0 arch=linux-rhel8-x86_64
[+]  45klwwn      ^builtin....@1.2.6%g...@8.4.0 arch=linux-rhel8-x86_64

***************     ^NOTE using builtin.libtirpc

[+]  tuhvokk          ^builti...@1.18.2%g...@8.4.0+shared patches=3d75052730690579484e65a5bf0f92f6c3b20d9c43a413862d087774f431d9e9 arch=linux-rhel8-x86_64
[+]  gfxnuzn              ^builtin...@1.1.1k%g...@8.4.0~docs+systemcerts arch=linux-rhel8-x86_64

I see that libtirpc from the builtin repo is being selected.  This does not change based on whether builtin.libtirpc is already installed or not.

If I do "spack --env gcc840 spec -lIN libtirpc", it returns fixed.umd.edu.libtirpc

If I do "spack --env gcc840 spec -lIN hdf ^libtirpc", then I also get fixed.umd.edu.libtirpc.

So I am really confused, why should

spack --env gcc840 spec -lIN hdf and

spack --env gcc840 spec -lIN hdf ^libtircp

produce different specs

Am I missing something really basic here?

FYI: The environment has

providers:

    rpc: [libtirpc]

The environment file also specifies a version (4.2.14) and some variants (+external-xdr +fortran ~java ~netcdf +pic ~shared +szip) for hdf, but nothing for libtirpc.

--

Tom Payerle
DIT-ACIGS/Mid-Atlantic Crossroads        pay...@umd.edu

5825 University Research Park               (301) 405-6135

University of Maryland
College Park, MD 20740-3831

[Gamblin, Todd]

Hi Thomas,

This seems like a bug — if you register a repo that overlays builtin, packages with identical names should be resolving to the highest precedence repository.

Are you using the new or old concretizer here?  We recently had a similar report of the old concretizer ignoring namespaces like this — the new concretizer fixed it in that case.

-Todd

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/CAHJ2ZQ8_aU6sfcSYWfNw9Fhi43u%3DB8hLR1%3D6XaQh_JecXv7GnQ%40mail.gmail.com.

[Pariksheet Nanda]

Hi Todd,

FYI - If I'm reading this correctly, Thomas was using the old concretizer and not the new 'clingo' solver:

On May 11, 2021, at 8:05 AM, Thomas M. Payerle <pay...@umd.edu> wrote:

I am using

> spack debug report

* **Spack:** 0.16.1-2429-f5e6c32495
* **Python:** 3.6.8
* **Platform:** linux-rhel8-x86_64
* **Concretizer:** original

Pariksheet

On 5/12/21 4:17 AM, 'Gamblin, Todd' via Spack wrote:

*Message sent from a system outside of UConn.*

To view this discussion on the web visit https://groups.google.com/d/msgid/spack/495C2CFA-C5CF-4966-974E-30760B936786%40llnl.gov.