Should clingo concretizer override preferences in spack|packages.yaml?

[Thomas M. Payerle]

About a month ago I began rebuilding our HPC software library to port to a new system.  As part of this, I updated spack to 0.17 and started using the new clingo concretizer. 

I have encountered a number of annoying differences in concretizer  behaviors between the new and old concretizers, and I am not sure if these are "features" or bugs.  Basically, when I specify package versions and variants in the spack.yaml file for an environment, the clingo concretizer seems to treat these specifications as only "suggestions".   I.e., I am not infrequently finding that spack "decides" to override my specifications when installing packages.  I am referring here to specifications in the environment's spack.yaml; specifications given on the command line do seem to be taken as "requirements", not "suggestions".

In some cases this is reasonable behavior, as I confess sometimes that my preferences are unsatisfiable (i.e I am trying to build foo@1 which depends on bar@:2 but I specified bar@3) --- IIRC for these cases the old concretizer would simply abort while trying to generate a spec (as does the clingo concretizer if I force the issue by adding the preference to the command line).

As an example,  consider the following spack.yaml for env tptest:

------------------------

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs: []
  view: false
  packages:
    all:
      compiler:: [g...@8.4.0]
    binutils:
      version: ['2.34']
      variants: ~gas +gold +headers +interwork ~ld +libiberty libs=shared,static +lto +nls +plugins

-----------------

I.e., I have requested binutils be built with variant '+gold'

But if I do 

-----------------

$ spack --env tptest spec binutils

Input spec
--------------------------------
binutils

Concretized
--------------------------------
binu...@2.34%g...@8.4.0~gas~gold+headers+interwork~ld+libiberty+lto+nls+plugins libs=shared,static arch=linux-rhel8-zen

[...]

---------------------------------

Note it concretized to ~gold

But if I do a spack --env tptest spec binutils+gold, it concretizes to '+gold'

(If I add -lI flags to the spec commands, the hashes of all of the dependencies

match for both cases).   So the '+gold' option is concretizable.

Why is spack ignoring my preferences?

--

Tom Payerle
DIT-ACIGS/Mid-Atlantic Crossroads        pay...@umd.edu

5825 University Research Park               (301) 405-6135

University of Maryland
College Park, MD 20740-3831

[Todd Gamblin]

Short answer: packages.yaml has always been a set of soft preferences, but the old concretizer was too dumb to really walk through them and find the “best” solution. 

So yes, the new one will try its best to do what you ask, but if it doesn’t, it almost certainly isn’t possible to satisfy all your preferences (the new concretizer is exhaustive).

That said, we’re actually working on an extension to packages.yaml for hard requirements, because it’s clearly useful.  That should hit develop pretty soon — see:

https://github.com/spack/spack/pull/27987

Todd

On Jan 10, 2022, at 8:01 AM, Thomas M. Payerle <pay...@umd.edu> wrote:



--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/CAHJ2ZQ8qgy2tW6J8jhF0gQxNusrQyQmHjmi1KNfVY%2B_%3DFaAMqw%40mail.gmail.com.