NVidia -> Khronos ray tracing refactor

[Robert Osfield]

Hi All,

One of the key items on my TODO list for VulkanScenGraph-1.0 is:

• Support for Khronos ray tracing.

I'm super lucky that Josef has already undertaken a port which is already work, though it's on a source code level fork of the VSG so I'll need to backport the changes.  I have branched Josef's VulkanPBRT repo so I can sync up changes with the main VulkanSceneGraph as I go.

    https://github.com/robertosfield/VulkanPBRT

I have also created KhronosRayTracing branches of the VSG:

   https://github.com/vsg-dev/VulkanSceneGraph/tree/KhronosRayTracing

and vsgExamples:

   https://github.com/vsg-dev/vsgExamples/tree/KhronosRayTracing

This branch don't have any commits yet, these branches are just laying the ground work, the actual merging work that I'll tackle tomorrow.

Cheers,

Robert.

[Robert Osfield]

I have now merged changes from Josef's https://github.com/robertosfield/VulkanPBRT repo and checked it into the VulkanSceneGraph KhronosRayTracing branch, the commit is:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/72bcf7ad783fd83b1a75d7280a66e31d558a64d2

I updated vsgExamples  KhronosRayTracing branch to build against these changes:

    https://github.com/vsg-dev/vsgExamples/commit/b62765de8535aa0d98f2f0028a8c0deaaf6b9b37

I was expecting to need to update the ray tracing shaders as they were built against the GL_NV_ray_tracing extension to GLSL, but it works for now. 

Cheers,

Robert.

[Robert Osfield]

After merging the changesfrom VulkanPBRT I've got the following compile warnings:

[ 72%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/BottomLevelAccelerationStructure.cpp.o

/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/BottomLevelAccelerationStructure.cpp: In member function ‘virtual void vsg::BottomLevelAccelerationStructure::compile(vsg::Context&)’:
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/BottomLevelAccelerationStructure.cpp:41:14: warning: unused variable ‘primitiveCount’ [-Wunused-variable]
    uint32_t primitiveCount = 0;
             ^~~~~~~~~~~~~~
[ 73%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/BuildAccelerationStructureTraversal.cpp.o
[ 73%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/DescriptorAccelerationStructure.cpp.o
[ 74%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/RayTracingPipeline.cpp.o
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/RayTracingPipeline.cpp: In constructor ‘vsg::RayTracingPipeline::Implementation::Implementation(vsg::Context&, vsg::RayTracingPipeline*)’:
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/RayTracingPipeline.cpp:167:30: warning: unused variable ‘memRequirements’ [-Wunused-variable]
        VkMemoryRequirements memRequirements;
                             ^~~~~~~~~~~~~~~
[ 75%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/RayTracingShaderGroup.cpp.o
[ 75%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/TopLevelAccelerationStructure.cpp.o
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/TopLevelAccelerationStructure.cpp: In member function ‘virtual void vsg::TopLevelAccelerationStructure::compile(vsg::Context&)’:
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/TopLevelAccelerationStructure.cpp:66:99: warning: missing initializer for member ‘VkBufferDeviceAddressInfo::pNext’ [-Wmissing-field-initializers]
    VkBufferDeviceAddressInfo bufferDeviceAddressInfo{VK_STRUCTURE_TYPE_BUFFER_DEVICE_ADDRESS_INFO};
                                                                                                  ^
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/TopLevelAccelerationStructure.cpp:66:99: warning: missing initializer for member ‘VkBufferDeviceAddressInfo::buffer’ [-Wmissing-field-initializers]
[ 76%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/TraceRays.cpp.o
[ 76%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/DrawMeshTasks.cpp.o

I will now look at this warnings.

Cheers,

Robert.

[Robert Osfield]

I have now fixed the warnings:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/4dd034051b1eaf2041a981172dd018ceddd651a6

[Robert Osfield]

I have update my VulkanSDK to 1.2.176.1 adn get the following errors:

[ 75%] Building CXX object src/vsg/CMakeFiles/vsg.dir/rtx/TopLevelAccelerationStructure.cpp.o

In file included from /home/robert/3rdParty/VulkanSDK/1.2.176.1/x86_64/include/vulkan/vulkan.h:11:0,
                from /home/robert/Dev/VulkanSceneGraph/include/vsg/commands/Command.h:15,
                from /home/robert/Dev/VulkanSceneGraph/include/vsg/commands/BindIndexBuffer.h:15,
                from /home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp:15:
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp: In constructor ‘vsg::AccelerationGeometry::AccelerationGeometry(vsg::Allocator*)’:
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp:29:61: error: cannot convert ‘std::nullptr_t’ to ‘VkDeviceAddress {aka long unsigned int}’ in assignment
    _geometry.geometry.triangles.vertexData.deviceAddress = VK_NULL_HANDLE;
                                                            ^
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp: In member function ‘void vsg::AccelerationGeometry::compile(vsg::Context&)’:
/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp:35:63: error: invalid operands of types ‘VkDeviceAddress {aka long unsigned int}’ and ‘std::nullptr_t’ to binary ‘operator!=’
    if (_geometry.geometry.triangles.vertexData.deviceAddress != VK_NULL_HANDLE) return;    // already compiled
                                                              ^

[Robert Osfield]

On Wed, 12 May 2021 at 14:13, Robert Osfield <robert....@gmail.com> wrote:

/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp: In constructor ‘vsg::AccelerationGeometry::AccelerationGeometry(vsg::Allocator*)’:

/home/robert/Dev/VulkanSceneGraph/src/vsg/rtx/AccelerationGeometry.cpp:29:61: error: cannot convert ‘std::nullptr_t’ to ‘VkDeviceAddress {aka long unsigned int}’ in assignment
    _geometry.geometry.triangles.vertexData.deviceAddress = VK_NULL_HANDLE;

I've checked in a fix:

   https://github.com/vsg-dev/VulkanSceneGraph/commit/0d68ec5bb2345ab18b421150505a0c3b370aed08

 

[Robert Osfield]

I have been testing the Khronos Ray Tracing port against different VulkanSDK versions and found that the vsgmeshshader example works with VulkanSDK-1.2.162.0 but fails with a VulkanSDK-1.2.176.1.  Khronos ray tracing works with both though..

I haven't yet come across anyone else hitting up against issues with mesh shaders and 1.2.176.1.  It looks to be a SPIR-V header issues, so the vsgExamples/data/shaders/meshshader.mesh compilation by the VSG/glsLang combo is producing SPIR-V that my driver doesn't like.

I don't know yet whether this is regression in glsLang in VulkanSDK-1.2.176.1 or some standard change, or a driver mismatch.  Has anyone else come across reports of issues?  My google searches haven't uncovered an clues?

I'm stumped and frustrated.  Ray tracing looks to be OK though...

Cheers,

Robert.

[Robert Osfield]

Hi Josef,

As a test of the port of the Khronos Ray Tracing changes from VulkanPBRT to VulkanSceneGraph I've forked VulkanPBRT and removed the local sources for VulkanSceneGraph, vsgImGui and vsgXchange, tweaked the CMakeLists.txt to pull in the external dependencies and it builds and runs as before :-)

I've put all these changes into a UseSeparateVulkanSceneGraph branch:

   https://github.com/robertosfield/VulkanPBRT/tree/UseSeparateVulkanSceneGraph

The branch requires the KhronosRayTracing branch of the VSG:

    https://github.com/vsg-dev/VulkanSceneGraph/compare/KhronosRayTracing

I would appreciate it if you can have a look through the KhronosRayTracing branch changes to see if everything looks complete.   If everything looks OK I'll merge this branch with master - I'll leave this till tomorrow though so yourself and others can test/pitch in.

If you'd like to adopt the changes I've made to VulkanPBRT just let me know I can general a PR on your main report.

I haven't looked at the Vulkan debug error:

Objects: 1
       [0] 0x4c000000004c, type: 9, name: NULL
UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout(ERROR / SPEC): msgNum: 1303270965 - Validation Error: [ UNASSIGNED-CoreValidation-DrawState-InvalidImageLayout ] Object 0: handle = 0x560d315aeb78, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID =
0x4dae5635 | Submitted command buffer expects VkImage 0x40000000004[] (subresource: aspectMask 0x1 array layer 0, mip level 0) to be in layout VK_IMAGE_LAYOUT_GENERAL--instead, current layout is VK_IMAGE_LAYOUT_UNDEFINED.
   Objects: 1
       [0] 0x560d315aeb78, type: 6, name: NULL

I think this is likely just a case of setting the image correctly.

Cheers,
Robert.

[Josef Stumpfegger]

Hi Robert,

nice i am going to take a look at it tomorrow morning and report if any errors occur or anything looks supicious.

Cheers,

Josef

[Josef Stumpfegger]

Hi Robert,

I have just checked out all the chnages and imported them into my project and everything works fine. I also have looked through all the relevant files and found everything to be as i changed it. I am now starting to implement other features as i already described and am going to report if vsg raytracing has to be expanded to suit more complex work. I will still use the source directly and build it with my project until i have finished the more complex work as debugging the library is easier this way.

Cheers,

Josef

[Robert Osfield]

Hi Josef,

On Thu, 13 May 2021 at 07:54, 'Josef Stumpfegger' via vsg-users : VulkanSceneGraph Developer Discussion Group <vsg-...@googlegroups.com> wrote:

I have just checked out all the chnages and imported them into my project and everything works fine. I also have looked through all the relevant files and found everything to be as i changed it. I am now starting to implement other features as i already described and am going to report if vsg raytracing has to be expanded to suit more complex work. I will still use the source directly and build it with my project until i have finished the more complex work as debugging the library is easier this way.

At some point in the future I would like to revisit the vsgFramework repo that I created very early in the VSG project as an experiment with compiling multiple projects together.  I'd hope we can provide a template that allows folks to more easily manage compile of all the VSG components they depend upon and enable custom versions of the software without needing to manually copy code back and forth to keep things in sync.

For now though I'd like to shake down the VSG's new KhronosRayTracing branch in prep for merging it with master. This morning I updated the the VulkanSceneGraph ShaderCompiler code to enable it target the SPIR-V version required when compile KHR ray racing shaders:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/5b6b6658a7ee2e9e11f89f4af35ce0e5d21a7265

I also updated the shaders in vsgExamples to use the KHR EXT extensions rather than NV versions:

   https://github.com/vsg-dev/vsgExamples/commit/258e9d4bc6717833f81b29f878d5312acfbc41f9

With this change I believe you should be able to use the GLSL shaders directly in your code rather than needing to compile to SPIR-V.  You should also be able to use vsgconv.

This change to targeting SPIR-V 1.4 by default is one that I expect I'll need to revisit and my need to add a SPIR-V version hint into vsg::ShaderModule to help guide vsg::ShaderCompiler/glsLang what version to target.

Cheers,

Robert.

[Robert Osfield]

Now that we have Khronos ray tracing extensions supported and they replace the previous RTX raytracing extensions I have moved the raytracing classes into include/vsg/raytracing and src/vsg/raytracing headers, leaving the mesh shader class in include/vsg/rtx and src/vsg/rtx.  The required changes are provided by two commits:

https://github.com/vsg-dev/VulkanSceneGraph/commit/a5cc6d94addcbdb344f0927059f55848a8523a11

https://github.com/vsg-dev/VulkanSceneGraph/commit/a340e18f4332bd69956236025d9e4441801a0ac4

Cheers,

Robert.

[Josef Stumpfegger]

Hi Robert,

The idea to add version specification for the shader compiler is defeniteley a good idea.

However i still prefer the precompiled version created with glslang on compilation of the project, as it removes the need of compiling the shader every time the program is started. This would mainly be useful if the shader code changes often while the program is running with a shader editor for example, but i dont intend to create one (maybe in the future).

Further i might in the next weeks add a few optimizations to the vsg raytracing. The Acceleration structures for example are built currently with the worst case size, while after building one could reduce the size fo the acceleration structure to the real space required and thus free up some memory. This will include a new postBuildAccelerationCommands list in the context to support this. Also making pipeline caching is of high importance, as it provides a significant speedup for pipeline creation.

Cheers,

Josef

[Robert Osfield]

Hi Josef,

On Thu, 13 May 2021 at 14:58, 'Josef Stumpfegger' via vsg-users : VulkanSceneGraph Developer Discussion Group <vsg-...@googlegroups.com> wrote:

The idea to add version specification for the shader compiler is defeniteley a good idea.

I think I'll need to come up with some solution for this before merging the KhronosRayTracing branch.

 

However i still prefer the precompiled version created with glslang on compilation of the project, as it removes the need of compiling the shader every time the program is started. This would mainly be useful if the shader code changes often while the program is running with a shader editor for example, but i dont intend to create one (maybe in the future).

Back in the early days of the VSG project I did consider having the SPRV cached on disk, with automatic recompilation when the glsl version was updated.  I thought about this even before I started writing any code.  Such a scheme would avoid the need to recompile on every run.  So far we haven't needed so I haven't attempted adding an extra layer of complexity.

My current approach with example is to write them using GLSL then convert to .spv when necessary.

 

Further i might in the next weeks add a few optimizations to the vsg raytracing. The Acceleration structures for example are built currently with the worst case size, while after building one could reduce the size fo the acceleration structure to the real space required and thus free up some memory. This will include a new postBuildAccelerationCommands list in the context to support this. Also making pipeline caching is of high importance, as it provides a significant speedup for pipeline creation.

I will have to re-read all the docs on ray tracing, though I'm feeling a bit thinly stretched across too many different VSG/Vulkan topics as I jump between various aspects of getting things nearer to beta status before 1.0.  Suggestions on improvements are welcome.  Real-world usage is particularly helpful to highlight issues of usability/flexibility and performance.

Cheers,

Robert.