Advice on finding packages (cuda) with the external package spack feature

[Jim Galarowicz]

Hi Matt, all,

I'm trying to find the cuda installation using the external packages interface, but spack can't seem to find it.
I've tried depends_on(cuda) as well as depends_on(cuda@6.0) and played with the settings inside packages.yaml, but so far nothing works.
So, I thought maybe I'd ask for help.

Thanks,
Jim G

spack install cbtf-argonavis
==> Error: Package cuda not found.

packages.yaml contents:

packages:
   openmpi:
      paths:
         ope...@1.10.2%g...@4.8.3=Linux-x86_64: /opt/openmpi-1.10.2
   cuda:
      paths:
         cuda@6.0%g...@4.8.3=Linux-x86_64: /usr/local/cuda-6.0
         cuda%g...@4.8.3=Linux-x86_64: /usr/local/cuda-6.0
   qt3:
      paths:
         q...@3.3.8%g...@4.8.3=Linux-x86_64: /usr/lib64/qt-3.3


cbtf-argonavis/package.py file contents:

from spack import *

class CbtfArgonavis(Package):
    """CBTF Argo Navis project contains the CUDA collector and supporting
       libraries that was done as a result of a DOE SBIR grant."""
    homepage = "http://sourceforge.net/p/cbtf/wiki/Home/"

    # Mirror access template example
    url      = "file:/home/jeg/OpenSpeedShop_ROOT/SOURCES/cbtf-argonavis-1.6.tar.gz"
    version('1.6', '0fafa0008478405c2c2319450f174ed4')

    #version('1.6', branch='master', git='http://git.code.sf.net/p/cbtf-argonavis/cbtf-argonavis')

    depends_on("cm...@3.0.2:")
    depends_on("bo...@1.57.0")
    depends_on("papi")
    depends_on("mr...@5.0.1")
    depends_on("cbtf")
    depends_on("cbtf-krell")
    depends_on("cuda")

    parallel = False
    optimization_setting = "-g -O2"
    build_type = 'None'

    def install(self, spec, prefix):

       # Look for package installation information in the cbtf and cbtf-krell prefixes
       cmake_prefix_path = join_path(spec['cbtf'].prefix) + ':' + join_path(spec['cbtf-krell'].prefix)

       with working_dir('CUDA'):
         with working_dir('build', create=True):
           cmake('..',
                 '-DCMAKE_BUILD_TYPE=%s'          % build_type,
                 '-DCMAKE_CXX_FLAGS=%s'           % optimization_setting,
                 '-DCMAKE_C_FLAGS=%s'               % optimization_setting,
                 '-DCMAKE_INSTALL_PREFIX=%s'    % prefix,
                 '-DCMAKE_PREFIX_PATH=%s'         % cmake_prefix_path,
                 '-DCUDA_DIR=%s'                          % spec['cuda'].prefix,
                 '-DCUPTI_DIR=%s'                          % join_path(spec['cuda'].prefix + '/extras/CUPTI'),
                 '-DCUPTI_ROOT=%s'                       % join_path(spec['cuda'].prefix + '/extras/CUPTI'),
                 '-DPAPI_ROOT=%s'                         % spec['papi'].prefix,
                 '-DCBTF_DIR=%s'                           % spec['cbtf'].prefix,
                 '-DCBTF_KRELL_DIR=%s'                % spec['cbtf-krell'].prefix,
                 '-DBOOST_ROOT=%s'                     % spec['boost'].prefix,
                 '-DBoost_DIR=%s'                          % spec['boost'].prefix,
                 '-DBOOST_LIBRARYDIR=%s'            % spec['boost'].prefix.lib,
                 '-DMRNET_DIR=%s'                        % spec['mrnet'].prefix,
                 '-DBoost_NO_SYSTEM_PATHS=ON',
                 *std_cmake_args)

           make("clean")
           make()
           make("install")



[jeg@localhost cbtf-argonavis]$ lsr /usr/local/cuda
0 lrwxrwxrwx. 1 root root 8 Oct 14  2014 /usr/local/cuda -> cuda-6.0
[jeg@localhost cbtf-argonavis]$ lsr /usr/local/cuda-6.0
total 60
4 drwxr-xr-x.  3 root root 4096 Apr  1  2014 targets
4 -rw-r--r--.  1 root root  365 Apr  1  2014 README
4 -rw-r--r--.  1 root root  365 Apr  1  2014 LICENSE
0 lrwxrwxrwx.  1 root root   28 Oct 14  2014 include -> targets/x86_64-linux/include
0 lrwxrwxrwx.  1 root root   24 Oct 14  2014 lib64 -> targets/x86_64-linux/lib
4 drwxr-xr-x.  8 root root 4096 Oct 14  2014 nvvm
4 drwxr-xr-x.  4 root root 4096 Oct 14  2014 open64
4 drwxr-xr-x.  2 root root 4096 Oct 14  2014 src
4 drwxr-xr-x.  8 root root 4096 Oct 14  2014 libnsight
4 drwxr-xr-x.  7 root root 4096 Oct 14  2014 libnvvp
4 drwxr-xr-x.  4 root root 4096 Oct 14  2014 extras
4 drwxr-xr-x.  2 root root 4096 Oct 14  2014 tools
4 drwxr-xr-x.  3 root root 4096 Oct 14  2014 bin
4 drwxr-xr-x. 13 root root 4096 Oct 14  2014 .
4 drwxr-xr-x. 12 root root 4096 Oct 14  2014 samples
4 drwxr-xr-x.  5 root root 4096 Oct 14  2014 doc
4 drwxr-xr-x. 17 root root 4096 Dec 21 14:12 ..

[Matthew LeGendre]

Externals can only point at packages that Spack has package.py entries
for. So you could add a package in:

spack/var/spack/repos/builtin/packages/cuda/package.py

Then use your package.yaml to say that cuda is external in
/usr/local/cuda-6.0. Your package.py need not correctly install cuda
toolkit if you're always going to use it external (though it probably
should do as much as it can before you commit a cudatoolkit package.py).

-Matt

> --
> You received this message because you are subscribed to the Google Groups "Spack" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
> To post to this group, send email to sp...@googlegroups.com.
> Visit this group athttps://secure-web.cisco.com/1yMHtOBw9M0Ge8ewOhOE1fj6djRiJVIMp6qOCUfrp1K-i-ax4vs_Gk6zsWwXDlMjIAxWmyoNMPiEjl28aHDYCfQqBVnwUBHVwlAT
> hw2xX-DZV8lcySdDpgun4LIQX_jDY5YHMg5f9RMUY_uaqxhEyQRV9rG9CLPdVJ3eLiXN8w6tdALl4S9S4tkdlGjKvCUmJOQ0NrbUbbI_7Ii0u_vqxvirz-RAwTH0geBEQ
> nQ56f1CDx-g8T4K8s5T6BE00VN6eABGU8am49xIiJW0hZ-yt0qwT85xeFuRmtu5Fj4nhu7bG3Mj6a2ohFHKyHiiZsy2UPPnH5xM4a5YyjEv2mFOxRAV_1ySlDuVGmn4Rd
> 9kpceFflz79HZ4behnPO21jCkjj/https%3A%2F%2Fgroups.google.com%2Fgroup%2Fspack.
> For more options, visithttps://secure-web.cisco.com/1KG_9dtseuCQO2lRtCpMjl1GDy4DP-7UDUWnn-IbsMCv3kYczuAeEpL3cGWEGnLSp1iDFmvxruCG4njc-uRteyagcWcTkR9NZb_j
> MIfGH6A0gJzyEFWEiVSaxXaKBIBiGBWfVshWjjVHSNUFV94HRxvpDzAv74TaBLpay5z9YAgWLEvU7MhHs5choJo7o0uawMP1o9Q4BYP2mMsXjfDrsH5rkjCgbRMsxs7Xa
> 2HstXYOPCK1yggT_o7nwSCUdeI6Ls5PyM2I0UK63MhJyN1-8-ksTxKmSNoc2nZqsGB39r9vOQR0-WcRe9ukBZu8nF3TCxRpG-JkQ2d-BW-4lwVnGA-ZZgClXUHmCEA8HC
> kRZgbF3AszoumI7bbqEwESGnN_x/https%3A%2F%2Fgroups.google.com%2Fd%2Foptout.
>
>

[Jim Galarowicz]

Hi Matt,

Thanks for the information! I appreciate it!

Jim G

[Jim Galarowicz]

Hi Matt, all,

I created a dummy cuda/package.py file, but there is no source code available to build cuda, as far as I know.
So, I dummied up a cuda tar file and put it in a mirror.
When I try to build cbtf-argonavis, spack seems to be in a loop.

Here is the dummy package.   I just created the cuda directory manually and copied in the cbtf package.py file and hand modified it.
Maybe the rest of spack doesn't know about the cuda package and that is causing the problem.
But, there isn't a tarball to point to for "spack create <tarball>"

Has anyone else tried to create a package file that is "dummied" up as a place holder for using an external package?

Any help is appreciated.

Thanks,
Jim G

DUMMY CUDA package.py:

from spack import *

class Cuda(Package):
    """CUDA tools."""
    homepage = "https://developer.nvidia.com/cuda-toolkit"

    # Mirror access template example

    url      = "file:/home/jeg/OpenSpeedShop_ROOT/SOURCES/cuda-6.0.tar.gz"
    version('6.0', '41e7f8f7ec22b152d8f470dbf23a539d')

    # Use when the git repository is available
    #version('1.6', branch='master', git='http://git.code.sf.net/p/cbtf/cbtf')

    parallel = False

    def install(self, spec, prefix):

      with working_dir('build', create=True):
          cmake('..',

        '-DCMAKE_INSTALL_PREFIX=%s'   % prefix,

                *std_cmake_args)

          make("clean")
          make()
          make("install")

WHERE WE APPEAR TO HANG:

19818 ?        00:12:23 firefox
20686 ?        00:00:00 gvfsd-http
29641 pts/1    00:00:29 python
30264 pts/3    00:00:00 ps
32605 pts/3    00:00:00 bash
[jeg@localhost SOURCES]$ !gdb
gdb -pid 29641


no debugging symbols found)...done.
Loaded symbols for /lib64/libuuid.so.1
0x0000003db9a7efdf in PyDict_GetItem () from /lib64/libpython2.7.so.1.0
Missing separate debuginfos, use: debuginfo-install python-2.7.5-15.fc19.x86_64
(gdb) where
#0  0x0000003db9a7efdf in PyDict_GetItem () from /lib64/libpython2.7.so.1.0
#1  0x0000003db9a6d2fd in PyFrame_New () from /lib64/libpython2.7.so.1.0
#2  0x0000003db9add6f8 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#3  0x0000003db9adebbd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#4  0x0000003db9a6dbf0 in ?? () from /lib64/libpython2.7.so.1.0
#5  0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#6  0x0000003db9a58515 in ?? () from /lib64/libpython2.7.so.1.0
#7  0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#8  0x0000003db9a9eac9 in ?? () from /lib64/libpython2.7.so.1.0
#9  0x0000003db9a9eba8 in ?? () from /lib64/libpython2.7.so.1.0
#10 0x0000003db9a851f9 in PyObject_RichCompare () from /lib64/libpython2.7.so.1.0
#11 0x0000003db9a8533f in PyObject_RichCompareBool () from /lib64/libpython2.7.so.1.0
#12 0x0000003db9a73bca in ?? () from /lib64/libpython2.7.so.1.0
#13 0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#14 0x0000003db9ad3a07 in ?? () from /lib64/libpython2.7.so.1.0
#15 0x0000003db9addc2e in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#16 0x0000003db9add74c in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#17 0x0000003db9adebbd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#18 0x0000003db9a6dbf0 in ?? () from /lib64/libpython2.7.so.1.0
#19 0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#20 0x0000003db9a58515 in ?? () from /lib64/libpython2.7.so.1.0
#21 0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#22 0x0000003db9ad8a37 in PyEval_CallObjectWithKeywords () from /lib64/libpython2.7.so.1.0
#23 0x0000003db9a9eccd in ?? () from /lib64/libpython2.7.so.1.0
#24 0x0000003db9a979df in ?? () from /lib64/libpython2.7.so.1.0
#25 0x0000003db9ad51ac in ?? () from /lib64/libpython2.7.so.1.0
#26 0x0000003db9addbae in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#27 0x0000003db9adebbd in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#28 0x0000003db9a6dbf0 in ?? () from /lib64/libpython2.7.so.1.0
#29 0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#30 0x0000003db9a58515 in ?? () from /lib64/libpython2.7.so.1.0
#31 0x0000003db9a49db3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) quit
A debugging session is active.

[Jim Galarowicz]

Hi all,

I had some other thoughts on this.

If I want to build OpenSpeedShop on NASA's Pleiades system and want to point the build at an external MPT MPI library install or Mvapich MPI on another system.   It seems I have to pollute the spack packages directory with a dummied up mpt and mvapich directory and package file.  I'm not interested in building those package, merely pointing to the external versions.   Is there a way around the "dummying up" of these.  I'm having the same issue with cuda.   Still stuck on that one.

Thanks,
Jim G

[Elizabeth F]

Have you looked at the results of PR #120?

Visit this group at https://groups.google.com/group/spack.
For more options, visit https://groups.google.com/d/optout.

[Todd Gamblin]

External package docs (added in #120) are here:

http://software.llnl.gov/spack/site_configuration.html#external-packages

[Jim Galarowicz]

Yes, I looked at the documentation, but it forces dummy directories and package files, even if you don't want to build it.
If you say buildable: False
It will fail if you don't have a sub-directory and package file.
That is what I'm seeing.

[Todd Gamblin]

The package file effectively reserves a name for the external package, so we ought to have package files in place for these things anyway.  Technically we don't *need* it now, but in the future, the externals support might do some compatibility checks on externals' dependencies (sot that they do not conflict with what is built) and in that case you'd definitely want the package file to describe to Spack how the dependencies should be constrained.

So I don't really mind cluttering up the package namespace with mvapich and MPT.  That's what you have to do at the moment.

That said, it could be useful to have "anonymous" externals that provide vdeps.  I haven't thought through how this would work, but you could imagine having something like:

packages:

mpi:

paths:

mpt@2.5%gcc:/path/to/mpt-install

Where you don't need to have a package file for mpt.  It makes me nervous, though, as Spack really wants to know what `mpt` is in order to merge it into a spec DAG.  If there is no placeholder for it, then some other package called mpt could come along with screwy dependencies and break things, once we add support for that.

Thoughts?  My inclination is that there should at least be a descriptor of some sort for anything Spack installs or links to, and package files are how we do that right now.

-Todd

[Jim Galarowicz]

Hi Todd,

I can try to put place holder package files and see how that works.
For some reason, my cuda attempt isn't working.  How far does the package.py have to go?   There is no source available for commercial packages like cuda.

Thanks,
Jim G

[Todd Gamblin]

Jim:

You can get a minimal boilerplate `package.py` with `spack edit -f <packagename>`.  I you do that, for now, make the install method raise an exception, e.g.:

from spack import *                                                                                                                         

from spack.package import InstallError                                                                                                      

                                                                                                                                            

class Cuda(Package):                                                                                                                        

    """Description"""                                                                                                                       

                                                                                                                                            

    homepage = "http://www.example.com"                                                                                                     

    url      = "http://www.example.com/cuda-1.0.tar.gz"                                                                                     

                                                                                                                                            

    version('1.0', '0123456789abcdef0123456789abcdef')                                                                                      

                                                                                                                                            

    def install(self, spec, prefix):                                                                                                        

        raise InstallError('CUDA must be used as an external package.') 

We should probably add something to make it easier to fail the build this way, as Erik noted here:

https://github.com/LLNL/spack/issues/294

[Jim Galarowicz]

Hi Todd,

Thanks.  I will try that.

Jim G