Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc

[David Mertens]

Hello everybody -

I recently resuscitated CUDA::Minimal, a module that provides basic memory allocation and memory transfer functionality for CUDA (i.e. video card) parallel programming. Note, you must write your own CUDA kernels separately; I recommend writing them in XS wrapper code, using ExtUtils::nvcc and can answer questions if you're interested.

When using ExtUtils::nvcc, your code gets compiled through a C++ compiler. I seem to have struck a problem with how XS handles its dVAR and dXSARGS declarations in the Boot section and in other sections. The applicable error can be found here: https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C code that leads to this error (generated from the XS code on my machine) is given in this gist: https://gist.github.com/run4flat/4974702.

The problem is a linkage conflict in the declaration for Perl__notused on lines 163 and 329, which I presume is embedded in dVAR or dXSARGS. However, I'm left scratching my head as to why the linkage would differ between these two. Based on my understanding of how extern "C" works, there is no difference that I can see. That is, there is no difference in the linkage specification for the two, because the extern "C" on line 369 does not have curly braces and therefore only effects the declaration of the function definition. It should not impact the declarations of any variables.

Anybody have any idea what's causing this linkage specification problem, and how I might fix it?

Thanks!
David

--
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan

[bulk88]

David Mertens wrote:

Hello everybody -

I recently resuscitated CUDA::Minimal, a module that provides basic memory allocation and memory transfer functionality for CUDA (i.e. video card) parallel programming. Note, you must write your own CUDA kernels separately; I recommend writing them in XS wrapper code, using ExtUtils::nvcc and can answer questions if you're interested.

When using ExtUtils::nvcc, your code gets compiled through a C++ compiler. I seem to have struck a problem with how XS handles its dVAR and dXSARGS declarations in the Boot section and in other sections. The applicable error can be found here: https://github.com/run4flat/perl-CUDA-Minimal/issues/7. The actual C code that leads to this error (generated from the XS code on my machine) is given in this gist: https://gist.github.com/run4flat/4974702.

The problem is a linkage conflict in the declaration for Perl__notused on lines 163 and 329, which I presume is embedded in dVAR or dXSARGS. However, I'm left scratching my head as to why the linkage would differ between these two. Based on my understanding of how extern "C" works, there is no difference that I can see. That is, there is no difference in the linkage specification for the two, because the extern "C" on line 369 does not have curly braces and therefore only effects the declaration of the function definition. It should not impact the declarations of any variables.

Anybody have any idea what's causing this linkage specification problem, and how I might fix it?

Thanks!
David

I think dVAR becomes dNOOP which is http://perl5.git.perl.org/perl.git/blob/f6fe275c937ceb508cdcfd033ce162e1fa01989e:/perl.h#l354. Note a C++ and a C version of that macro. Can you post the preprocessed version of the .c file or have you looked at the preprocessed version for clues as to what happened?

[David Mertens]

Sorry, meant to reply-all.

David

---------- Forwarded message ----------
From: David Mertens <dcmerte...@gmail.com>
Date: Mon, Feb 18, 2013 at 12:51 AM
Subject: Re: Apparent linkage conflicts with dVAR; dXSARGS; when compiled using nvcc
To: bulk88 <bul...@hotmail.com>

bulk88 -

I've updated the original gist with a new file that includes the direct C(++) code as preprocessed by nvcc. The two look identical to me. Should I try to bump-up the verbosity of the compiler?

David

--
 "Debugging is twice as hard as writing the code in the first place.
  Therefore, if you write the code as cleverly as possible, you are,
  by definition, not smart enough to debug it." -- Brian Kernighan

[David Mertens]

Sorry for the rapid-fire. I'm up late and the ideas are only trickling out.

It appears that the code doesn't think __cplusplus is defined. That is odd. nvcc claims (or, it used to) that it always runs the code through a C++ compiler at the end of the day, so why it wouldn't have __cpluplus defined is beyond me.

David

[bulk88]

David Mertens wrote:
> Sorry for the rapid-fire. I'm up late and the ideas are only trickling
> out.
>
> It appears that the code doesn't think __cplusplus is defined. That is
> odd. nvcc claims (or, it used to) that it always runs the code through
> a C++ compiler at the end of the day, so why it wouldn't have
> __cpluplus defined is beyond me.
>
> David

Compiler flag on cmd line? file name auto detection (.c vs .cc or .cpp)?
Read the compilers manual or google it. Also your XS module needs to use
PERL_NO_GET_CONTEXT for efficiency.

your file is called "Minimal.c", per
http://sbel.wisc.edu/Courses/ME964/2008/Documents/nvccCompilerInfo.pdf
your file will be compiled in regular C mode. Try a (untested)

#ifndef __cplusplus
#error c++ is not defined
#endif

for debugging.

[David Mertens]

bulk88 -

nvcc knows about C, C++, and CUDA-C. I do not recall if it differentiates between C and C++, though. I know that I correctly specify that Minimal.c should be compiled as CUDA-C, but I'm not sure if I told it to do that in the preprocessor command that produced the code I included in the above gist. I will have to check when I get home tonight, which will be in 8 hours or so.

As for the #ifndef error idea for debugging, that's perfect. I'll give it a run and see what it produces.

David

[David Mertens]

Sorry for not getting back to this as quickly as I had hoped. My machine with nVidia hardware is at home and I haven't had much time in the las few evenings to play with it. This may be the case through the weekend; we shall see.

bulk88, I was not entirely sure what PERL_NO_GET_CONTEXT did. After some digging, I came across this article that seemed to clarify things quite a bit, at least for me: http://perl.active-venture.com/pod/perlguts-concurreny.html. I also did some reading and discovered that there are ways to have thread-local data in C, which was news to me. (I never learned the intricacies of C.) Suddenly the behavior of dTHX; and the whole api with the _nocontext suffix make so much more sense.

Thanks!
David

On Mon, Feb 18, 2013 at 6:34 AM, bulk88 <bul...@hotmail.com> wrote:

[bulk88]

David Mertens wrote:
> Sorry for not getting back to this as quickly as I had hoped. My
> machine with nVidia hardware is at home and I haven't had much time in
> the las few evenings to play with it. This may be the case through the
> weekend; we shall see.
>
> bulk88, I was not entirely sure what PERL_NO_GET_CONTEXT did. After
> some digging, I came across this article that seemed to clarify things
> quite a bit, at least for me:
> http://perl.active-venture.com/pod/perlguts-concurreny.html. I also
> did some reading and discovered that there are ways to have
> thread-local data in C, which was news to me. (I never learned the
> intricacies of C.) Suddenly the behavior of dTHX; and the whole api
> with the _nocontext suffix make so much more sense.
>
> Thanks!
> David
>
>

PERL_NO_GET_CONTEXT removed a bazillion func calls to Perl_get_context
or in your case (since you are posix os), pthread_getspecific. This
makes your XS code much faster (20 to 100% faster). Look at the
preprocessed version of the .c file at
https://gist.github.com/run4flat/4974702 . Sadly PERL_NO_GET_CONTEXT's
biggest reason to use it (performance) isn't in perlguts. Its main
purpose it to remove the extra get context func calls. The thread local
data stuff (
http://perldoc.perl.org/perlxs.html#Safely-Storing-Static-Data-in-XS )
few modules use (they normally get_sv a package SV), but there is no
reason why you can't use it.