CUDA support

[Nils Moehrle]

What is the best way to support CUDA projects?

CUDA, as a language, is essentially C++ with extensions. During compilation nvcc (Nvidia CUDA Compiler) is compiling all gpu specific code and replaces all non C++ constructs with valid C++ the result is than handed over to a standard C++ compiler (usually gcc).

Because of this I originally thought it would be sufficient to add a nvcc toolset to premake.

So I basically copied clang.lua and prefixed some flags that nvcc does not understand with -Xcompiler (this flag tells nvcc to hand over the next flag to the C++ compiler).

And there the problems started: .cu files are not handed to nvcc because path.iscppfile does not, and is right to do so, classify .cu files as C++ files.

To get an idea of what other problems will arise I just patched path.iscppfile to accept .cu files.

So far just one other issue arose: the dependency generation in makefiles is currently hardcoded as -MF "$(@:%%.gch=%%.d)" one of the flags nvcc does not understand.

There is a similar flag for nvcc:

> nvcc --help

    ...

    --dependency-target-name <target>          (-MT)                                                                               

            Specify the target name of the generated rule when generating a dependency                                             

            file (see '--generate-dependencies').

    ...

Changing the flag results in a successful build :-)

However, nvcc is generating a lot of other dependency files within the build directory. So far I haven't figured out how to specify their location.

Which brings me back to the original question - after reading this, do you think adding nvcc as a toolset is a valid approach to support CUDA projects?

And if so would somebody be willing to help me finalizing this (like answering some questions via chat).

The "Adding a New Toolset" section in the wiki couldn't explain everything to me ;-)

Thanks,

Nils

[Tom van Dijck]

> Because of this I originally thought it would be sufficient to add a nvcc toolset to premake.

Yeah I think that should typically work, the problem is a little that most of premake isn't as "plugable" as you would like in this situation, and so a lot falls on overriding the right methods etc.. I did recently change the way the visual studio action deals with 'different' extensions in this commit: https://github.com/premake/premake-core/commit/cf1b44018cf61f3af018035e86e173ec7d10049e

but that is obviously only something that works for Visual Studio, and has no effect on the xcode or gmake backend. so depending on the platform you are targetting things might have to be done in different ways.

> The "Adding a New Toolset" section in the wiki couldn't explain everything to me ;-)

Documentation is hard, and any feedback towards improvements are very welcome.

> And if so would somebody be willing to help me finalizing this (like answering some questions via chat).

Sure thing, I think google hangouts might work for me, or skype... just email me and we'll work something out...

[Nils Moehrle]

Digging a little further...

It turned out that the dependency files that have been generated within the build directory were all generated by gcc during the compilation of intermediate files because I was handing over the -MMD and -MP flags to gcc via -Xcompiler.

Furthermore I completely missundestood the -MT option, it just changes the filename within the dependency file...

The bad news is that nvcc is apparently unable to generate dependencies and compile files at the same time (at least my version - CUDA compilation tools, release 7.5, V7.5.17).

And while it is possible to stop gcc after preprocessing (-E > /dev/null)  this would degrade the performance so I think one would have to alter the makefile generator.

It is also important to note that the dependency generation of nvcc has no equivalents to -MMD (create only dependencies to non-system headers) and -MP (create phony targets for headers).

Regards,

Nils

[Nils Moehrle]

The premake documentation is great!
That was a mere justification why I would appreciate help. In fact the adding a toolkit is about the only thing that I came across that is not documented.

Best,

Nils

PS. Mail is out :-)

[starkos]

I don't have too much to add to this—it sounds like you're on the right track—except to say that we're open to changes and suggestions on improving the process. I think we all agree that the toolset approach in particular need improvement.

[Nils Moehrle]

I created a pull request such that you can have a look how invasive the changes are.

Are you in general interested in cuda support in premake-core or should this go into a module?

I could resolve almost all issues with the makefile generator, I could not check any of the others.

One remaining issue is that some flags are now given to the cpp compiler twice, once via -Xcompiler and once via the nvcc internal flag conversion.
This results in warnings because some of the flags are only valid for the cpp compiler but some of the intermediate files are c.
Is there a possibility to remove the host compiler c++11 flag if the appropriate flag is already supplied by nvcc? (in getcxxflags(cfg)) 

I would appreciate it if you could give me some comments on the pull request.

Thanks,
Nils

PS. Where should I post minor issues within the wiki? e.g. table.isempty is twice in the Lua libary additions 

[starkos]

> .cu files are not handed to nvcc because path.iscppfile does not, and is right to do so, classify .cu files as C++ files.

A thought: what if instead of calling `path.iscppfile()` we called `tool.canbuild()` or something similar? That way each tool could do its own checking, even if it just calls out to `path.iscppfile()` internally.

[Nils Moehrle]

Good idea, that would make it more flexible and might lead to better (earlier) errors.

Now that I think of it is there a check that each file within a project has been handled? I think I had the issue before that some .cu file were just ignored during the makefile creation.
It would be nice to give out warnings like `no buildrule could be created for file "foo.xyz"` or `file "foo.xyz" is not supported by toolset "bar"`.

Also, I have not tested what happens if one feeds a '.m' file to nvcc but I guess it would not like it. Those things could be prevented that way.

[starkos]

> is there a check that each file within a project has been handled?

No. If a file doesn't get handled, it is because Premake thought it was part of a configuration that was not included in the project. Since the check to see if the files were handled would need to perform the same check, it wouldn't actually do anything.

If you can find a case where this happens, open an issue with a simple test case that shows the problem and we'll get it patched up.

> I have not tested what happens if one feeds a '.m' file to nvcc but I guess it would not like it

Another good reason to move the "should build this file" check to the toolset. Should have thought of that sooner!

Jason

[Nils Moehrle]

I am still trying to come up with a clean way to let the host compiler handle only flags that nvcc does not know.
The possibilities that I see right now are:
- remove all flags known to nvcc from the config and reinsert them afterwards.
- duplicate the config and remove the all flags to nvcc
- remove flags that are known to nvcc from the flags the host compiler produced
- change the `config.mapFlags` function to accept multiple mappings which are searched in order and giving it for example first the `nvcc.cxxflags` and then the `hostcompiler.cxxflags`

Performance wise the last option would probably be the best but I would also mean the largest change to the existing code base...

What are your thoughts on this?

Nils

PS. Are you guys on the #premake channel on freenode?

[Nils Moehrle]

I have implemented the first of the mentioned possibilities in this commit

https://github.com/nmoehrle/premake-core/commit/946e44c196f2bac1a4f52d44d00b98c40f01c94f

What do you think?

Cheers,

Nils

[starkos]

Sorry for the delayed response. Another possibility would be to base the nvcc flag tables on a copy of the GCC versions, and then overlay it with your new values. Something like:

m.cflags = table.merge(gcc.cflags, {

flags = {

RelocatableDeviceCode = "--relocatable-device-code=true",

},

-- and so on

})

The downside is that it takes a snapshot of the GCC flag tables and won't catch any updates that might happen later.

It might be nice to have a simple "inherits" function that uses Lua metatables to allow a table to dynamically inherit values from another table on the fly (basically, if it isn't in your table, return whatever is in the "base" table).

[Nils Moehrle]

I think I prefer this method over the one I implemented :-)

I would however require that each host compiler will continue to provide the flags (cflags, cxxflags and ldflags) and it would also restrict the get*flags to this behavior:

    function gcc.get*flags(cfg)                                                 

        local flags = config.mapFlags(cfg, gcc.*flags)                 

        return flags                                                            

    end                                                                         

The special case with cflags, where the warnings are joined with the cflags, would have to treated differently within the nvcc toolset.

The prefixing might also be a little bit trickier because some entries within the table could be functions, but I think it would work.

This prefixing also kind of ruins the elegant approach of a inherits function, although one could optionally have a function as parameter which is applied to inherited values...

[Nils Moehrle]

I have implemented your proposed method in this commit https://github.com/nmoehrle/premake-core/commit/608343eb24d8abdf6199b39daf7ff2d3393b100c

It does look cleaner in comparison to the methods I implemented earlier, at least to me.

However, I didn't manage to implement and leverage the table.inhert function...

What do you think about this implementation?

[starkos]

This looks fine to me!

[Nils Moehrle]

I think there is a problem with msc though, since msc has different flags depending on the linkage mode `iif(cfg.kind ~= premake.STATICLIB, msc.linkerFlags, msc.librarianFlags)`
Further it adds some flags within getcflags and getldflags so in this stage the patch wouldn't work on windows.