Re: [spack] Problem installing MXNet

111 views
Skip to first unread message

Elizabeth A. Fischer

unread,
May 19, 2018, 8:31:36 AM5/19/18
to Grigori Fursin, Spack
`jpeg` is a virtual package that can be satisfied by `libjpeg` or `libjpeg-turbo`.  The problem here is that one package is requesting `libjpeg` by name, and another `libjpeg-turbo`.  Almost certainly, both should be requesting `jpeg`.

I see that
  1. mxnet -> opencv -> vtk -> libjpeg


I have no idea where you're picking up `libjpeg-turbo`; again, is this a configuration file issue?  Or maybe an interal file?  Maybe this file is messing you up:

./etc/spack/defaults/packages.yaml:      jpeg: [libjpeg-turbo, libjpeg]



In any case... `vtk/package.py` should almost certainly depend on `jpeg`, not `libjpeg`.  That is the bug.

-- Elizabeth


r-tiff/package.py:    depends_on("libjpeg")

virtualgl/package.py:    depends_on("libjpeg-turbo")

vtk/package.py:    depends_on('libjpeg')





 
==============================================================
fursin@velociti:~$ spack install mxnet

==> Error: Multiple providers found for 'jpeg': ['libjpeg', 'libjpe...@1.5.90%gcc@6.3.0 arch=linux-ubuntu16.04-x86_64 ^cm...@3.11.1%g...@6.3.0~doc+ncurses+openssl+ownlibs~qt arch=linux-ubuntu16.04-x86_64  ^nasm ^ncurses@6.0%g...@6.3.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64  ^ope...@1.0.2n%g...@6.3.0+systemcerts arch=linux-ubuntu16.04-x86_64  ^pkg...@1.4.0%g...@6.3.0 arch=linux-ubuntu16.04-x86_64 ^zl...@1.2.11%g...@6.3.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64']



--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+unsubscribe@googlegroups.com.
To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
For more options, visit https://groups.google.com/d/optout.

Grigori Fursin

unread,
May 19, 2018, 10:13:36 AM5/19/18
to elizabet...@columbia.edu, Spack
Hi Elizabeth,

Unfortunately, I don't yet understand internals or configuration files of spack and just wanted to try it out of the box with default settings to see how it installs a few packages like mxnet which we use for our R&D.

I didn't pick up libjpeg-turbo or any other deps - I just downloaded spack on a clean system and immediately tried to install mxnet. My understanding of spack is that it can work out of the box to "virtually" install a given tool with default deps. These are steps which I did on a new clean laptop (where spack was not installed before). Maybe it's a bug in the mxnet package itself?

fursin@fgg-t470p:~/spack$ git clone https://github.com/spack/spack

Cloning into 'spack'...
remote: Counting objects: 114828, done.
remote: Compressing objects: 100% (88/88), done.
remote: Total 114828 (delta 59), reused 52 (delta 24), pack-reused 114707
Receiving objects: 100% (114828/114828), 38.34 MiB | 2.81 MiB/s, done.
Resolving deltas: 100% (55487/55487), done.

fursin@fgg-t470p:~/spack$ cd spack/bin

fursin@fgg-t470p:~/spack/spack/bin$ ./spack install mxnet

==> Error: Multiple providers found for 'jpeg': ['libjpeg', 'libjpe...@1.5.90%g...@6.3.0 arch=linux-ubuntu16.04-x86_64 ^cm...@3.11.1%g...@6.3.0~doc+ncurses+openssl+ownlibs~qt arch=linux-ubuntu16.04-x86_64  ^nasm ^ncurses@6.0%g...@6.3.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64  ^ope...@1.0.2n%g...@6.3.0+systemcerts arch=linux-ubuntu16.04-x86_64  ^pkg...@1.4.0%g...@6.3.0 arch=linux-ubuntu16.04-x86_64 ^zl...@1.2.11%g...@6.3.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64']

Thanks,
Grigori

Elizabeth A. Fischer

unread,
May 19, 2018, 12:32:27 PM5/19/18
to Grigori Fursin, Spack
Grigori,

Spack does a lot of great things, but unfortunately it is not magic.  It's basically >2000 recipes of how to build specific packages, plus a concretization algorithm that choose appropriate versions of those packages to work together.

We do try to make Spack packages install by default "out of the box."  But unfortunately, we do not have the ability to continuously test all 2,000+ packages to ensure they install by default on any given system.  When people submit packages, we look over them to ensure they are reasonable.  Certainly, all packages should concretize by default, and we hope to be able to auto-test that someday.

Spack is a kind of "install collective", where you benefit from the efforts of others figuring out how to build things.  I need a stack of 100 packages; and using Spack is MUCH easier than figuring out how to build them all myself.  In my experience, if I come back to Spack after a while and things have changed (the OS, the computer, Spack itself), typically ~5% of packages have some kind of problem, which I have to fix.  Look at recent PR's by @citibeth, and you will see about 10 packages that worked on my CentOS7 system --- but then I uncovered bugs installing on SLES11!

Not surprisingly, more common packages are more likely to "just work" due to their larger user base.  You chose two packages with a small user base, and uncovered two bugs somewhere.  

The other thing Spack offers is a community to help find and fix problems when you do encounter them.  Based on the information you've given, I hope to be able to replicate both the bugs you found this weekend, which will likely result in some bug fixes in short order.

-- Elizabeth

Grigori Fursin

unread,
May 19, 2018, 1:50:22 PM5/19/18
to elizabet...@columbia.edu, Spack, Gamblin, Todd, Anton Lokhmotov, Leo Gordon, Flavio Vella
Elizabeth,

I hope you didn't get me wrong - I am very interested in spack project exactly because of the reasons you mentioned and I think it's a really great concept! I totally understand that it's very difficult to keep all possible packages up-to-date across lots of platforms and environments and it requires collaborative effort (I was suffering from similar problems during past 15 years ;) )! That's why I am glad if my feedback can help fix a few issues. I am also sorry that I can't yet help myself since I didn't yet have time to dive deeply inside spack.

This collaborative concept is also very much in line with our own Collective Knowledge project (CK) where we built reusable and portable workflows particularly for AI/ML (that's why I am interested in tensorflow/caffe/spack). We also rely on the community to fix various components of such workflows which run across different platforms: : https://github.com/ctuning/ck

Obviously packaging is our weakest link (building Caffe, Caffe2, TensorFlow and MXNet is awfully complex across continuously changing software/hardware stack) - that's why we developed our own package manager to solve current packaging mess and allow packages to mix with already installed software. But it is not really our main focus - we have to focus on implementing and improving high-level workflows such as AI/ML benchmarking and SW/HW co-design. We now have good support for TensorFlow, Caffe, Caffe2, MXNet, CNTK, but we lack support of many other packages. That's why I was very glad to find out about spack! I think we can combine CK workflows with spack packages and I just wanted to check how to install caffe or mxnet via spack as a starting point ;) ...

Please, tell me if you will be able to fix caffe or mxnet but please take your time - it's not urgent. If it works out, I can then try to integrate them with CK workflows and will send a usability report here. It can be also useful to bootstrap discussions for our upcoming workshop on portable HPC workflows at SC'18: http://rescue-hpc.org

Thanks a lot,
Grigori

sche...@llnl.gov

unread,
May 24, 2018, 2:44:53 PM5/24/18
to Spack
Installing mxnet specifically runs up against a few complexities in Spack. If you are willing to try out PRs, you can see

https://github.com/spack/spack/pull/8162

In which case, the following may work for you: "spack install mxnet ^libjpeg ^prot...@3.1.0 ^vtk+osmesa"

If you have a system-installed opengl and are willing to experiment with externals, you can try

"spack install mxnet ^libjpeg ^prot...@3.1.0 ^opengl"

The short story is that the logic which assembles the full dependency DAG is greedy and can paint itself into a corner (the linked PR also has some more details on that). The "^foo" type arguments are hints to the concretizer to help it along.

In the longer term, the concretizer will be improved to look ahead and avoid issues like this.

Elizabeth is also right that many of these issues may be caused by package definitions overly-constraining their dependencies (e.g. asking for a specific version when really they could use any version newer than X) and updating the package definitions directly would help reduce (and in many cases could eliminate) these problems. That being said in terms of someone new to Spack editing package definitions isn't necessarily the very first thing one wants to do before testing out installing things for the first time.

Grigori Fursin

unread,
May 26, 2018, 8:27:29 AM5/26/18
to sche...@llnl.gov, Spack
Hi,

I patched my spack installation with this PR 8162 and it solved the problem - thanks!

I still didn't manage to complete mxnet installation via spack since there were some other issues in other dependencies (OpenCV?), but I now understand better what to do next: I prepared spack configuration yaml to use CUDA from a non-standard path, and I also tried to play with different dependencies for mxnet:

$ spack install mxnet ^libjpeg ^ll...@5.0.1 ^pyt...@3.5.1 ^ope...@3.1.0 ^prot...@3.1.0 ^vtk+osmesa

So now I see where to dig further, if needed ;) . Thank you for your explanations and help - very appreciated!
Grigori
--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

Elizabeth A. Fischer

unread,
May 26, 2018, 10:29:22 AM5/26/18
to Grigori Fursin, Scheibel, Peter J., Spack
Grigori,

If you have further problems... I VERY MUCH recommend you try placing your constraints in a `packages.yaml` file instead of on the command line.  In that past, that has been more reliable.  The recent PR might have fixed it; but if you have trouble concretizing, it's still something to try.

-- Elizabeth

To unsubscribe from this group and stop receiving emails from it, send an email to spack+unsubscribe@googlegroups.com.

To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+unsubscribe@googlegroups.com.

Elizabeth A. Fischer

unread,
May 26, 2018, 10:40:34 AM5/26/18
to Grigori Fursin, Scheibel, Peter J., Spack
Grigori,

See here, hopefully we can get this kind of stuff cleaned up over the long run:


-- Elizabeth


On Sat, May 26, 2018 at 8:27 AM, Grigori Fursin <gfu...@gmail.com> wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to spack+unsubscribe@googlegroups.com.

To post to this group, send email to sp...@googlegroups.com.
Visit this group at https://groups.google.com/group/spack.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+unsubscribe@googlegroups.com.

Grigori Fursin

unread,
May 27, 2018, 10:26:06 AM5/27/18
to elizabet...@columbia.edu, Scheibel, Peter J., Spack
Thank you, Elizabeth, for this hint.

I checked 'packages.yaml' and it is indeed quite easy to customize from our  Collective Knowledge benchmarking workflows - I just created a prototype CK workflow with integrated spack where I automatically generate this file and then install required dependencies for our benchmarks via spack (such as LLVM compiler). I just need to fix a few minor issues and I can then share this use case with the spack community.

Cheers,
Grigori

Grigori Fursin

unread,
May 27, 2018, 10:35:04 AM5/27/18
to elizabet...@columbia.edu, Scheibel, Peter J., Spack
That's very reasonable, Elizabeth!

I agree that it will be really very convenient for end-users to make packages work out-of-the-box - I lead a project on customizable benchmarking and optimization of real workloads, and our users expect to run shared programs on any platform and with any environment out of the box with default configurations. Only as a second step, we allow users to customize installation: change libraries, compilers, tools or try different sets of optimizations, etc...

Cheers,
Grigori

Jimmy Tang

unread,
Jun 11, 2018, 4:57:39 PM6/11/18
to Spack
I ran into this a while back, as far as I recall the mapping was cached somewhere, it is possible cleanup the cache and let spack recalculate this information


On Saturday, 19 May 2018 13:31:36 UTC+1, Elizabeth A. Fischer wrote:
`jpeg` is a virtual package that can be satisfied by `libjpeg` or `libjpeg-turbo`.  The problem here is that one package is requesting `libjpeg` by name, and another `libjpeg-turbo`.  Almost certainly, both should be requesting `jpeg`.

I see that
  1. mxnet -> opencv -> vtk -> libjpeg


I have no idea where you're picking up `libjpeg-turbo`; again, is this a configuration file issue?  Or maybe an interal file?  Maybe this file is messing you up:

./etc/spack/defaults/packages.yaml:      jpeg: [libjpeg-turbo, libjpeg]



In any case... `vtk/package.py` should almost certainly depend on `jpeg`, not `libjpeg`.  That is the bug.

-- Elizabeth


r-tiff/package.py:    depends_on("libjpeg")

virtualgl/package.py:    depends_on("libjpeg-turbo")

vtk/package.py:    depends_on('libjpeg')





 
==============================================================
fursin@velociti:~$ spack install mxnet

==> Error: Multiple providers found for 'jpeg': ['libjpeg', 'libjpe...@1.5.90%gcc@6.3.0 arch=linux-ubuntu16.04-x86_64 ^cm...@3.11.1%g...@6.3.0~doc+ncurses+openssl+ownlibs~qt arch=linux-ubuntu16.04-x86_64  ^nasm ^ncurses@6.0%g...@6.3.0~symlinks~termlib arch=linux-ubuntu16.04-x86_64  ^ope...@1.0.2n%g...@6.3.0+systemcerts arch=linux-ubuntu16.04-x86_64  ^pkg...@1.4.0%g...@6.3.0 arch=linux-ubuntu16.04-x86_64 ^zl...@1.2.11%g...@6.3.0+optimize+pic+shared arch=linux-ubuntu16.04-x86_64']




--
You received this message because you are subscribed to the Google Groups "Spack" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.

Grigori Fursin

unread,
Jun 14, 2018, 5:15:44 AM6/14/18
to Jimmy Tang, Spack
Hi Jimmy,
I tired to install it on a clean system in Docker and it still didn't work - I think there is a mix up in a configuration of one of the packages as Elizabeth suggested.
Cheers,
Grigori

Jimmy Tang

unread,
Jun 14, 2018, 6:01:51 AM6/14/18
to Spack
Hey Grigori,

As far as I recall I removed " ~/.spack/cache/providers/builtin-index.yaml" then spack will regenerate the cache then things started working for me...

Jimmy

Jimmy Tang

unread,
Jun 14, 2018, 6:03:32 AM6/14/18
to Spack
More specifically


there is a spack clean --misc-cache which may be of use to you

Grigori Fursin

unread,
Jun 16, 2018, 10:55:37 AM6/16/18
to Jimmy Tang, Spack
Hi Jimmy,

Thank you for your suggestion - this indeed helped! However, mxnet installation now fails in VTK :( :

$ bin/spack clean --misc-cache

==> Removing cached information on repositories

$ bin/spack install mxnet ^libjpeg ^prot...@3.1.0 ^opengl

==> pkgconf is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/pkgconf
==> ncurses is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/ncurses
==> zlib is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/zlib
==> openssl is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/openssl
==> cmake is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/cmake
==> dmlc-core is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/dmlc-core
==> mshadow is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/mshadow
==> nnvm is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/nnvm
==> openblas is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/openblas
==> cu...@9.1.85 : externally installed in /usr/local/cuda-9.1.85
==> cu...@9.1.85 : already registered in DB
==> yasm is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/yasm
==> ffmpeg is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/ffmpeg
==> bzip2 is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/bzip2
==> xz is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/xz
==> libxml2 is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxml2
==> tar is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/tar
==> gettext is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/gettext
==> libffi is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libffi
==> pcre is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/pcre
==> readline is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/readline
==> gdbm is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/gdbm
==> perl is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/perl
==> sqlite is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/sqlite
==> python is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/python
==> glib is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/glib
==> libsigsegv is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libsigsegv
==> m4 is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/m4
==> bison is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/bison
==> util-macros is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/util-macros
==> font-util is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/font-util
==> libpng is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libpng
==> freetype is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/freetype
==> gperf is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/gperf
==> fontconfig is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/fontconfig
==> inputproto is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/inputproto
==> kbproto is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/kbproto
==> libpthread-stubs is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libpthread-stubs
==> xproto is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/xproto
==> libxau is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxau
==> libxdmcp is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxdmcp
==> xcb-proto is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/xcb-proto
==> libxcb is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxcb
==> xextproto is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/xextproto
==> xtrans is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/xtrans
==> libx11 is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libx11
==> libxext is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxext
==> renderproto is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/renderproto
==> libxrender is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxrender
==> pixman is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/pixman
==> cairo is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/cairo
==> help2man is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/help2man
==> flex is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/flex
==> gobject-introspection is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/gobject-introspection
==> atk is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/atk
==> libjpeg is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libjpeg
==> libtiff is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libtiff
==> gdk-pixbuf is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/gdk-pixbuf
==> icu4c is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/icu4c
==> harfbuzz is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/harfbuzz
==> libxft is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libxft
==> pango is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/pango
==> libbsd is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libbsd
==> expat is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/expat
==> perl-xml-parser is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/perl-xml-parser
==> intltool is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/intltool
==> shared-mime-info is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/shared-mime-info
==> gtkplus is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/gtkplus
==> jasper is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/jasper
==> jdk is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/jdk
==> libtool is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libtool
==> libpciaccess is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libpciaccess
==> autoconf is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/autoconf
==> automake is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/automake
==> numactl is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/numactl
==> hwloc is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/hwloc
==> openmpi is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/openmpi
==> protobuf is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/protobuf
==> py-setuptools is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/py-setuptools
==> py-numpy is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/py-numpy
==> ope...@4.5.0 : externally installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/opengl
==> ope...@4.5.0 : already registered in DB
==> glew is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/glew
==> hdf5 is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/hdf5
==> jsoncpp is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/jsoncpp
==> libharu is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/libharu
==> lz4 is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/lz4
==> netcdf is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/netcdf
==> netcdf-cxx is already installed in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/netcdf-cxx
==> Installing vtk
==> Using cached archive: /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/cache/vtk/vtk-8.0.1.tar.gz
==> Staging archive: /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1.tar.gz
==> Created stage in /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n
==> No patches needed for vtk
==> Building vtk [CMakePackage]
==> Executing phase: 'cmake'
==> Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j20'

5 errors found in build log:
     1686    cd /tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/exodusII/vtkexodusII && /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/lib/spack
             /env/gcc/gcc -DVTK_IN_VTK -DvtkexoIIc_EXPORTS -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/exodusII -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-d
             emo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/exodusII -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-bu
             ild/ThirdParty/netcdf -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/netcdf -
             I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/opengl/include -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/hdf5 -I/home/fursin/CK
             -TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/hdf5 -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-d
             emo-spack-linux-64/spack/hdf5/include -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/zlib/include -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux
             -64/spack/openmpi/include -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/zlib -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/v
             ar/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/zlib -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-
             7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/exodusII/vtkexodusII/include -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/exodusII/vtkex
             odusII/include  -w -w -O2 -g -DNDEBUG -fPIC   -o CMakeFiles/vtkexoIIc.dir/exgtt.c.o   -c /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0
             .1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/exodusII/vtkexodusII/exgtt.c
     1687    [  4%] Building C object ThirdParty/oggtheora/vtkoggtheora/CMakeFiles/vtkoggtheora.dir/libtheora-1.1.1/lib/encode.c.o
     1688    cd /tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/oggtheora/vtkoggtheora && /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/lib/spa
             ck/env/gcc/gcc -DHAVE_CONFIG_H -DVTK_IN_VTK -Dvtkoggtheora_EXPORTS -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/oggtheora -I/home/fursin/CK-TO
             OLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/oggtheora  -w -w -O2 -g -DNDEBUG -fPIC   -o CMakeF
             iles/vtkoggtheora.dir/libtheora-1.1.1/lib/encode.c.o   -c /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fux
             ha6n/VTK-8.0.1/ThirdParty/oggtheora/vtkoggtheora/libtheora-1.1.1/lib/encode.c
     1689    [  4%] Building C object ThirdParty/oggtheora/vtkoggtheora/CMakeFiles/vtkoggtheora.dir/libtheora-1.1.1/lib/enquant.c.o
     1690    cd /tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/oggtheora/vtkoggtheora && /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/lib/spa
             ck/env/gcc/gcc -DHAVE_CONFIG_H -DVTK_IN_VTK -Dvtkoggtheora_EXPORTS -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/oggtheora -I/home/fursin/CK-TO
             OLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/oggtheora  -w -w -O2 -g -DNDEBUG -fPIC   -o CMakeF
             iles/vtkoggtheora.dir/libtheora-1.1.1/lib/enquant.c.o   -c /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fu
             xha6n/VTK-8.0.1/ThirdParty/oggtheora/vtkoggtheora/libtheora-1.1.1/lib/enquant.c
     1691    CMakeFiles/vtkgl2ps.dir/gl2ps.c.o: In function `gl2psConvertPixmapToPNG':
  >> 1692    /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/gl2ps/vtkgl2ps/gl2ps.c:834: unde
             fined reference to `png_set_longjmp_fn'
  >> 1693    collect2: error: ld returned 1 exit status
     1694    ThirdParty/gl2ps/vtkgl2ps/CMakeFiles/vtkgl2ps.dir/build.make:94: recipe for target 'lib/libvtkgl2ps-8.0.so.1' failed
  >> 1695    make[2]: *** [lib/libvtkgl2ps-8.0.so.1] Error 1
     1696    make[2]: Leaving directory '/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build'
     1697    CMakeFiles/Makefile2:4907: recipe for target 'ThirdParty/gl2ps/vtkgl2ps/CMakeFiles/vtkgl2ps.dir/all' failed
  >> 1698    make[1]: *** [ThirdParty/gl2ps/vtkgl2ps/CMakeFiles/vtkgl2ps.dir/all] Error 2
     1699    make[1]: *** Waiting for unfinished jobs....
     1700    [  4%] Building C object ThirdParty/oggtheora/vtkoggtheora/CMakeFiles/vtkoggtheora.dir/libtheora-1.1.1/lib/fdct.c.o
     1701    cd /tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/oggtheora/vtkoggtheora && /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/lib/spa
             ck/env/gcc/gcc -DHAVE_CONFIG_H -DVTK_IN_VTK -Dvtkoggtheora_EXPORTS -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/oggtheora -I/home/fursin/CK-TO
             OLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/oggtheora  -w -w -O2 -g -DNDEBUG -fPIC   -o CMakeF
             iles/vtkoggtheora.dir/libtheora-1.1.1/lib/fdct.c.o   -c /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha
             6n/VTK-8.0.1/ThirdParty/oggtheora/vtkoggtheora/libtheora-1.1.1/lib/fdct.c
     1702    [  4%] Building C object ThirdParty/exodusII/vtkexodusII/CMakeFiles/vtkexoIIc.dir/exgvan.c.o
     1703    cd /tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/exodusII/vtkexodusII && /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/lib/spack
             /env/gcc/gcc -DVTK_IN_VTK -DvtkexoIIc_EXPORTS -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/exodusII -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-d
             emo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/exodusII -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-bu
             ild/ThirdParty/netcdf -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/netcdf -
             I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/opengl/include -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/hdf5 -I/home/fursin/CK
             -TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/hdf5 -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-d
             emo-spack-linux-64/spack/hdf5/include -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/zlib/include -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux
             -64/spack/openmpi/include -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/zlib -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/v
             ar/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/zlib -I/home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-
             7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/exodusII/vtkexodusII/include -I/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/exodusII/vtkex
             odusII/include  -w -w -O2 -g -DNDEBUG -fPIC   -o CMakeFiles/vtkexoIIc.dir/exgvan.c.o   -c /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.
             0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/ThirdParty/exodusII/vtkexodusII/exgvan.c
     1704    [  4%] Building CXX object ThirdParty/verdict/vtkverdict/CMakeFiles/verdict.dir/V_KnifeMetric.cpp.o

     ...

     2306    /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/lib/spack/env/gcc/gcc -fPIC   -w -O2 -g -DNDEBUG  -Wl,--no-undefined -lc   -shared -Wl,-soname,libvtksqlite-8.0.
             so.1 -o ../../../lib/libvtksqlite-8.0.so.1 CMakeFiles/vtksqlite.dir/vtk_sqlite3.c.o -Wl,-rpath,::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
             ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: -lpthread
     2307    cd /tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build/ThirdParty/sqlite/vtksqlite && /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/spack/cmake/bin/c
             make -E cmake_symlink_library ../../../lib/libvtksqlite-8.0.so.1 ../../../lib/libvtksqlite-8.0.so.1 ../../../lib/libvtksqlite-8.0.so
     2308    make[2]: Leaving directory '/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build'
     2309    [ 16%] Built target vtksqlite
     2310    make[1]: Leaving directory '/tmp/fursin/spack-stage/spack-stage-Sge3Ux/VTK-8.0.1/spack-build'
     2311    Makefile:132: recipe for target 'all' failed
  >> 2312    make: *** [all] Error 2

See build log for details:
  /home/fursin/CK-TOOLS/mxnet-spack-cuda-demo-spack-linux-64/src/var/spack/stage/vtk-8.0.1-7nlu4jellsalcgffqfx5tksm4fuxha6n/VTK-8.0.1/spack-build.out
Error: cmake failed!
CK error: package installation failed!

Cheers,
Grigori

Jimmy Tang

unread,
Jun 28, 2018, 11:17:11 AM6/28/18
to Spack
I'm not too sure what to suggest on this, you might just need to update to the latest devel branch or go back in time till you find a commit that works. I installed mxnet a while back and it did work at somepoint.
Reply all
Reply to author
Forward
0 new messages