RTX Raytracing and major Viewer refactor

[Robert Osfield]

HI All,

Back in the 2019 Tom was working on adding RTX ray tracing support to the VSG, in December through to today I've picked up on this work and tackled Viewer refactoring work that makes advanced techniques like ray tracing fit more easily into the VSG.  This work has taken a little longer to wrap up due to me tackling the OpenSceneGraph-3.6.5 stable release work in December and January. 

Today I merged all this work with master, it changes the Viewer API in a number of places so you'll need to update your own applications/libs to use the new API.  The Pull Request containing all of Tom and my work is:

    https://github.com/vsg-dev/VulkanSceneGraph/pull/131

Note the github stats on the PR, 97 commits and 91 files changed, so we are talking about substantial changes.  In this thread I'll provide guidance on changes and pointers to what to look out when porting your own code base.

The vsgExamples set has also been updated, pulling together Tom's new example vsgraytracing example and my work on vsgdraw/vsgviewer/vsgcompute to utilize the new viewer related changes:

    https://github.com/vsg-dev/vsgExamples/pull/35

First up, some pictures, the obligatory first ray traced triangle - run the vsgraytracing from vsgExamples:

Next up ray tracing of a 3d model converted to the required top level/bottom level acceleration structures, run vsgtracing -m :

[Robert Osfield]

To see what changes you will need to make to your own applications/libs have a look at the changes to the vsgExamples example vsgviewer below, the key changes are replacing use of vsg::GraphicsStage with the new vsg::CommandGraph/RenderGraph, and calling new Viewer update(), recordAndSubmit() and present() methods to kick off the traversal and rendering:

diff --git a/Desktop/vsgviewer/vsgviewer.cpp b/Desktop/vsgviewer/vsgviewer.cpp

index 0304a4e..1d46019 100644
--- a/Desktop/vsgviewer/vsgviewer.cpp
+++ b/Desktop/vsgviewer/vsgviewer.cpp
@@ -167,18 +167,9 @@ int main(int argc, char** argv)
         if (maxPageLOD>=0) databasePager->targetMaxNumPagedLODWithHighResSubgraphs = maxPageLOD;
     }
 
-    // add a GraphicsStage to the Window to do dispatch of the command graph to the command buffer(s)
-    auto graphicsStage = vsg::GraphicsStage::create(vsg_scene, camera);
-    graphicsStage->databasePager = databasePager;
-    window->addStage(graphicsStage);
-
-    // compile the Vulkan objects
-    viewer->compile();
-
     // add close handler to respond the close window button and pressing escape
     viewer->addEventHandler(vsg::CloseHandler::create(viewer));
 
-
     if (pathFilename.empty())
     {
         viewer->addEventHandler(vsg::Trackball::create(camera));
@@ -198,18 +189,22 @@ int main(int argc, char** argv)
         viewer->addEventHandler(vsg::AnimationPathHandler::create(camera, animationPath, viewer->start_point()));
     }
 
+    auto commandGraph = vsg::createCommandGraphForView(window, camera, vsg_scene);
+    viewer->assignRecordAndSubmitTaskAndPresentation({commandGraph}, databasePager);
+
+    viewer->compile();
 
     // rendering main loop
     while (viewer->advanceToNextFrame() && (numFrames<0 || (numFrames--)>0))
     {
-        if (databasePager) databasePager->updateSceneGraph(viewer->getFrameStamp());
-
         // pass any events into EventHandlers assigned to the Viewer
         viewer->handleEvents();
 
-        viewer->populateNextFrame();
+        viewer->update();
+
+        viewer->recordAndSubmit();
 
-        viewer->submitNextFrame();
+        viewer->present();
     }
 
     // clean up done automatically thanks to ref_ptr<>

[Robert Osfield]

Changes to the VSG to be aware of:

The vsg::DispatchTraversal has been renamed vsg::RecordTraversal, this has been done to make the naming more consistent with the Vulkan. The use of the world dispatch is really aligned with OpenGL conventions of dispatch command and data into the OpenGL FIFO, while Vulkan usage has applications Recording command and data into command buffers, which are the Submitted to Queues.

Since Vulkan has multiple Queue types for different types work - be it Compute, Graphics or RayTracing and one needs to manage the recording to the command buffers that get submitted to these queues it made sense to encapsulate this work with a vsg::RecordAndSubmitTask that wraps up the vsg::CommandGraph that you wish to record in the CommandBuffers then Submit to the appropriate Queues.  Within each CommandBuffer you may also record command to one or more RenderPass, and this is now managed by nested one or more vsg::RenderGraph within each vsg::CommandGraph.

Once data has been submitted to the queues, when you have a window to present the result on you submit to the presentation queue using the new vsg::Presentation class.  The Viewer class provides some convenience methods for settings this all up for you.

    auto commandGraph = vsg::createCommandGraphForView(window, camera, vsg_scene);

    viewer->assignRecordAndSubmitTaskAndPresentation({commandGraph}, databasePager);

Alternatively you wire up the CommandGraph/RenderGraph/Presentation/RecordAndSubmitTask, but this does require you to handle all the queues and semaphores, consider this advanced usage :-)

[Robert Osfield]

The implementation of the vsg::createCommandGraphForView(..) convenience function mentioned above is pretty straight forward, it creates a CommndGraph that nested a RenderGraph that is configured to render to support Window and from the supplied Camera's view:

ref_ptr<CommandGraph> vsg::createCommandGraphForView(Window* window, Camera* camera, Node* scenegraph)

{

auto commandGraph = CommandGraph::create(window);

// set up the render graph for viewport & scene

auto renderGraph = vsg::RenderGraph::create();

renderGraph->addChild(ref_ptr<Node>(scenegraph));

renderGraph->camera = camera;

renderGraph->window = window;

renderGraph->renderArea.offset = {0, 0};

renderGraph->renderArea.extent = window->extent2D();

renderGraph->clearValues.resize(2);

renderGraph->clearValues[0].color = window->clearColor();

renderGraph->clearValues[1].depthStencil = VkClearDepthStencilValue{1.0f, 0};

commandGraph->addChild(renderGraph);

return commandGraph;

}

The convenience method Viewer:assignRecordAndSubmitTaskAndPresentation({commandGraph}, databasePager) then takes the CommandGraph's you want to traverse to record to command buffer(s) and then submit these to  graphics queue and wires it up the presentation using the required semaphore to coordinate the operations on the GPU:

void Viewer::assignRecordAndSubmitTaskAndPresentation(CommandGraphs commandGraphs, DatabasePager* databasePager) {

if (_windows.empty()) return;

// TODO need to resolve what to do about graphic vs non graphic operations and mulitple windows

Window* window = _windows.front();

Device* device = window->device();

PhysicalDevice* physicalDevice = window->physicalDevice();

auto [graphicsFamily, presentFamily] = physicalDevice->getQueueFamily(VK_QUEUE_GRAPHICS_BIT, window->surface());

auto renderFinishedSemaphore = vsg::Semaphore::create(device);

// set up Submission with CommandBuffer and signals

auto recordAndSubmitTask = vsg::RecordAndSubmitTask::create();

recordAndSubmitTask->commandGraphs = commandGraphs;

recordAndSubmitTask->signalSemaphores.emplace_back(renderFinishedSemaphore);

recordAndSubmitTask->databasePager = databasePager;

recordAndSubmitTask->windows = _windows;

recordAndSubmitTask->queue = device->getQueue(graphicsFamily);

recordAndSubmitTasks.emplace_back(recordAndSubmitTask);

presentation = vsg::Presentation::create();

presentation->waitSemaphores.emplace_back(renderFinishedSemaphore);

presentation->windows = _windows;

presentation->queue = device->getQueue(presentFamily);

}

In theory this new approach should enable us to handling multiple windows, multiple views within a window, or spread across windows as well as mixed compute, ray tracing and graphics operations.  It also opens the door to adding multi-threading for the recording of the command buffers where each vsg::RecordTraversal gets it's own thread to render it it's own vsg::CommandBuffer.

The new scheme should make it possible to handle multiple graphics cards, though that's require lots more changes to come, mostly internal so I don't think it'll affect the single GPU usage too much.