How to deal with 'nvcc' (cuda compiler)

[Rob]

Hi All,

I am trying to get meson to work with the nvidia cuda compiler (nvcc). The input file extensions are .cu and a native .o is output.

Right now, I have this:

cuda_gen = generator(nvcc,

    output: '@BASENAME@.o',

    arguments: ['-c', '@INPUT@', '-o', '@OUTPUT@'],

)

src = [

    'f1.cu',

    'f2.cu',

]

This fails...

generated = cuda_gen.process(src)

This works:

generated = cuda_gen.process('f1.cu')

Also:

lib1 = static_library('lib1', generated,

    include_directories: [cuda_inc, anotherlib_inc],

)

Unfortunately, the 'cuda_gen.process()' fails because the 'arguments' arg is not good enough, it needs a couple of '-Ipath/to/lib1' strings just like GCC.

Any advice on the best way to deal with this? Am I on the right track with this?

I'd really like to temporary define a compiler that acts like gcc that accepts '*.cu' files and calls 'nvcc'.

I've thought about setting up a cross compile file, but I don't think that helps me either, because this issue is only need for one library in this project.

Thanks!

[Rob]

I've made some progress on this. I've simply hard coded paths for the NVCC command for now and I can loop over the source like this:

    generated = []

    foreach s : cuda_src

        generated += cuda_gen.process(s)

    endforeach

The issue I have now is that when I add the 'generated' list to my exe, it complains.

app = executable('app', src, generated)

It says this:

ninja: error: '../apps/src/f1.cu', needed by 'apps/src/app@exe/f1.o', missing and no known rule to make it

I don't know how to tell meson to use the .o files that get generated for nvcc.

[Jussi Pakkanen]

On Thu, Nov 12, 2015 at 2:54 AM, Rob <rdool...@gmail.com> wrote:

 

src = [

    'f1.cu',

    'f2.cu',

]

This fails...

generated = cuda_gen.process(src)

This works:

generated = cuda_gen.process('f1.cu')

 

Weird, that should work just fine. We even have a test for that:

https://github.com/mesonbuild/meson/blob/master/test%20cases/common/29%20pipeline/meson.build

Are you setting up your pipeline in the exact same way as that one? Alternatively can you put your whole source tree viewable somewhere?

Unfortunately, the 'cuda_gen.process()' fails because the 'arguments' arg is not good enough, it needs a couple of '-Ipath/to/lib1' strings just like GCC.

You can set extra linker flags with the link_args keyword argument. 

[Rob]

Hi Jussi

On Thursday, November 12, 2015 at 2:02:08 PM UTC-8, Jussi Pakkanen wrote:

On Thu, Nov 12, 2015 at 2:54 AM, Rob <rdool...@gmail.com> wrote:

 

src = [

    'f1.cu',

    'f2.cu',

]

This fails...

generated = cuda_gen.process(src)

This works:

generated = cuda_gen.process('f1.cu')

 

Weird, that should work just fine. We even have a test for that:

https://github.com/mesonbuild/meson/blob/master/test%20cases/common/29%20pipeline/meson.build

I just did a test - when I had src = ['f1.cu'] it worked (as in your example), but as soon as I added another file to the list it failed.

Traceback (most recent call last):
  File "/opt/meson/share/meson/meson.py", line 208, in run
    app.generate()
  File "/opt/meson/share/meson/meson.py", line 161, in generate
    g.generate(intr)
  File "/opt/meson/share/meson/ninjabackend.py", line 145, in generate
    [self.generate_target(t, outfile) for t in self.build.get_targets().values()]
  File "/opt/meson/share/meson/ninjabackend.py", line 145, in <listcomp>
    [self.generate_target(t, outfile) for t in self.build.get_targets().values()]
  File "/opt/meson/share/meson/ninjabackend.py", line 203, in generate_target
    self.generate_custom_generator_rules(target, outfile)
  File "/opt/meson/share/meson/ninjabackend.py", line 1135, in generate_custom_generator_rules
    sole_output = os.path.join(self.get_target_private_dir_abs(target), outfilelist[i])

 

Are you setting up your pipeline in the exact same way as that one? Alternatively can you put your whole source tree viewable somewhere?

Unfortunately, the 'cuda_gen.process()' fails because the 'arguments' arg is not good enough, it needs a couple of '-Ipath/to/lib1' strings just like GCC.

You can set extra linker flags with the link_args keyword argument. 

The issue I had is my generator needed those '-Ixxx' lines. So now my generator looks like:

 cuda_gen = generator(nvcc,

    output: '@BASENAME@.o',

    arguments: ['-c', '@INPUT@', '-o', '@OUTPUT@', '-I', lib_inc_str, '-I', '/usr/local/nvidia/5.0/cuda/samples/common/inc/'],

 )

Is there a way to extract the string path from 'lib_inc = include_directories('.')'? 

My more pressing issue at the moment is the inability to use the generated objects when it comes time to build the final app.

[Jussi Pakkanen]

On Fri, Nov 13, 2015 at 12:30 AM, Rob <rdool...@gmail.com> wrote:

 

The issue I had is my generator needed those '-Ixxx' lines. So now my generator looks like:

 cuda_gen = generator(nvcc,

    output: '@BASENAME@.o',

    arguments: ['-c', '@INPUT@', '-o', '@OUTPUT@', '-I', lib_inc_str, '-I', '/usr/local/nvidia/5.0/cuda/samples/common/inc/'],

 )

Is there a way to extract the string path from 'lib_inc = include_directories('.')'? 

Include_directories is a bit more involved and does stuff behind the scenes so extracting is not so straightforward. However if you just need the source dir there is an easier way, like this:

'-I' + meson.current_source_dir()

 

My more pressing issue at the moment is the inability to use the generated objects when it comes time to build the final app.

A fix for this was just committed so it should work now.

 

[Rob]

Hi Jussi,

On Friday, November 13, 2015 at 8:37:08 AM UTC-8, Jussi Pakkanen wrote:

On Fri, Nov 13, 2015 at 12:30 AM, Rob <rdool...@gmail.com> wrote:

 

The issue I had is my generator needed those '-Ixxx' lines. So now my generator looks like:

 cuda_gen = generator(nvcc,

    output: '@BASENAME@.o',

    arguments: ['-c', '@INPUT@', '-o', '@OUTPUT@', '-I', lib_inc_str, '-I', '/usr/local/nvidia/5.0/cuda/samples/common/inc/'],

 )

Is there a way to extract the string path from 'lib_inc = include_directories('.')'? 

Include_directories is a bit more involved and does stuff behind the scenes so extracting is not so straightforward. However if you just need the source dir there is an easier way, like this:

'-I' + meson.current_source_dir()

 

Ok, that's what I'm doing.

 

My more pressing issue at the moment is the inability to use the generated objects when it comes time to build the final app.

A fix for this was just committed so it should work now.

Fix works - thanks! 

So, now that I have generated .o files from .cu files, how to I use them to build my final exe?

cuda_gen = generator(nvcc,

    output: '@BASENAME@.o',

    arguments: ['-c', '@INPUT@', '-o', '@OUTPUT@', '-I', lib1_inc_str, '-I', '/usr/local/nvidia/5.0/cuda/samples/common/inc/'],

)

cuda_src = ['f1.cu', 'f2.cu']

generated = cuda_gen.process(cuda_src)

This fails:

app = executable('backproject', src, generated)

output: ninja: error: '../f1.cu', needed by 'app@exe/f1.o', missing and no known rule to make it

This also fails

app = executable('backproject', src,

    objects: generated,

)

output: Bad object in target app.

This also fails:

app = executable('backproject', src,

    link_with: generated,

)

output: Link target is not library.

How do link with the object files that the generator outputs?

[Jussi Pakkanen]

On Fri, Nov 13, 2015 at 8:56 PM, Rob <rdool...@gmail.com> wrote:

 

So, now that I have generated .o files from .cu files, how to I use them to build my final exe?

This fails:

app = executable('backproject', src, generated)

output: ninja: error: '../f1.cu', needed by 'app@exe/f1.o', missing and no known rule to make it

This is the correct way to do it and we even have a test case that does exactly the same:

https://github.com/mesonbuild/meson/blob/master/test%20cases/common/59%20object%20generator/meson.build

Can you try that and see if it works for you?

Please check that the files you are trying to use (f1.cu) are actually in the location you have told Meson they are. Assuming from your output you have your .cu files at source root and your build dir is in the same dir. Meson should check that all input files exist but there may be bugs.

[Rob]

This fails:

app = executable('backproject', src, generated)

output: ninja: error: '../f1.cu', needed by 'app@exe/f1.o', missing and no known rule to make it

 

I've finally looked this error closer - it's not with meson, it's with ninja. It's telling me that f1.cu isn't there.. and sure enough, it's not. Please disregard my last message.

<facepalm>

Thanks a lot for your help!