Problems when mixing virtual packages and variants?

[Benedikt Hegner]

Hi all,

I am struggling with a strange problem with virtual packages and specs. I could bring it down to a "small" reproducer (*). Let's take a fake package that depends on both
depends_on("py-scipy")
depends_on("py-numpy")

In my RHEL6 setup that immediately results in an invalid spec error (*). Having a closer look we have the following dependency:
"py-scipy" -> "lapack" (a virtual package)
"py-numpy" -> "netlib-lapack" (a concrete package)

On resolving the virtual package "lapack" spack picks up the default variant (~shared), while "py-numpy" explicitly asks for the +shared variant. And hence the conflict. I guess that's not the desired behaviour of how virtual packages should be resolved?

In the minimal reproducer w/ py-scipy and py-numpy, I could get things running after adding the two following constrains:
depends_on("netlib-lapack+shared")
depends_on("netlib-blas+fpic")

Unfortunately I was hit by yet another feature that doesn't look that obvious. For this there is a second reproducer (**) using a toy package w/ the following dependencies:
depends_on("eigen")
depends_on("netlib-blas+fpic")
depends_on("netlib-lapack+shared")
depends_on("py-scipy")
depends_on("py-numpy")
depends_on("py-matplotlib")

I end up with a "Multiple providers found for 'lapack'" as shown below (**). It is particularly strange as adding/removing the totally unrelated eigen has an impact on how lapack gets resolved.

Any help or idea is appreciated.

Thanks,
Benedikt

(*)
git clone https://github.com/hegner/spack.git -b bug_virtualpackages
cd spack
./bin/spack install reproducer
==> Error: Invalid spec: 'netlib...@3.5.0%g...@4.4.7~shared=linux-x86_64^cm...@3.4.3%g...@4.4.7+ncurses=linux-x86_64^ncurses@6.0%g...@4.4.7=linux-x86_64^netli...@3.5.0%g...@4.4.7~fpic=linux-x86_64'. Package netlib-lapack requires variant +shared, but spec asked for ~shared

(**)
./bin/spack install reproducer2
==> Error: Multiple providers found for 'lapack': ['netlib-lapack+shared', 'netlib...@3.5.0%g...@4.4.7+shared=linux-x86_64^cm...@3.4.3%g...@4.4.7+ncurses=linux-x86_64^ncurses@6.0%g...@4.4.7=linux-x86_64^netli...@3.5.0%g...@4.4.7+fpic=linux-x86_64']

[Erik Schnetter]

I am surprised that py-numpy depends on netlib-lapack. If there's no
comment next to this specification explaining why, then I suspect an
error. You could try changing this to "lapack" instead.

-erik

> --
> You received this message because you are subscribed to the Google Groups "Spack" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
> To post to this group, send email to sp...@googlegroups.com.
> Visit this group at https://groups.google.com/group/spack.
> For more options, visit https://groups.google.com/d/optout.



--
Erik Schnetter <schn...@gmail.com>
http://www.perimeterinstitute.ca/personal/eschnetter/

[benedik...@cern.ch]

On Saturday, February 20, 2016 at 10:14:28 PM UTC+1, Erik Schnetter wrote:

I am surprised that py-numpy depends on netlib-lapack. If there's no
comment next to this specification explaining why, then I suspect an
error. You could try changing this to "lapack" instead.

Erik,

in the +blas variant (which is default for py-numpy) it does. Doesn't sound too strange to me.

  Benedikt
 

> ==> Error: Invalid spec: 'netlib...@3.5.0%g...@4.4.7~shared=linux-x86_64^cmake@3.4.3%g...@4.4.7+ncurses=linux-x86_64^ncurses@6.0%g...@4.4.7=linux-x86_64^netli...@3.5.0%g...@4.4.7~fpic=linux-x86_64'. Package netlib-lapack requires variant +shared, but spec asked for ~shared
>
> (**)
> ./bin/spack install reproducer2
> ==> Error: Multiple providers found for 'lapack': ['netlib-lapack+shared', 'netlib...@3.5.0%g...@4.4.7+shared=linux-x86_64^cmake@3.4.3%g...@4.4.7+ncurses=linux-x86_64^ncurses@6.0%g...@4.4.7=linux-x86_64^netli...@3.5.0%g...@4.4.7+fpic=linux-x86_64']

[benedik...@cern.ch]

Indeed, disabling it does solve the problem. Thanks for the suggestion as I can at least continue with my work for now.

Feels still like a bug though...

  Benedikt

[Erik Schnetter]

On Sat, Feb 20, 2016 at 4:19 PM, <benedik...@cern.ch> wrote:
>
>
> On Saturday, February 20, 2016 at 10:14:28 PM UTC+1, Erik Schnetter wrote:
>>
>> I am surprised that py-numpy depends on netlib-lapack. If there's no
>> comment next to this specification explaining why, then I suspect an
>> error. You could try changing this to "lapack" instead.
>
>
> Erik,
>
> in the +blas variant (which is default for py-numpy) it does. Doesn't sound
> too strange to me.

There is no reason to require Netlib's Lapack and Blas. In fact,
Netlib's Blas is known to be quite slow, and I don't see why this
should be the default. Spack has a beautiful mechanism for virtual
packages, and people might want to use other Lapack or Blas
implementations, such as e.g. MKL or Atlas or OpenBlas.

-erik

>> > 'netlib...@3.5.0%g...@4.4.7~shared=linux-x86_64^cm...@3.4.3%g...@4.4.7+ncurses=linux-x86_64^ncurses@6.0%g...@4.4.7=linux-x86_64^netli...@3.5.0%g...@4.4.7~fpic=linux-x86_64'.

>> > Package netlib-lapack requires variant +shared, but spec asked for ~shared
>> >
>> > (**)
>> > ./bin/spack install reproducer2
>> > ==> Error: Multiple providers found for 'lapack':
>> > ['netlib-lapack+shared',

>> > 'netlib...@3.5.0%g...@4.4.7+shared=linux-x86_64^cm...@3.4.3%g...@4.4.7+ncurses=linux-x86_64^ncurses@6.0%g...@4.4.7=linux-x86_64^netli...@3.5.0%g...@4.4.7+fpic=linux-x86_64']