vsgdynamicload

[Denis Taniguchi]

Hi guys,

Quick question: is vsgdynamicload example working for you?

Just want to make sure there is nothing wrong with my env. I am using
latest master for everything.

Cheers,

Denis

[Denis Taniguchi]

Oh, forgot to mention I am trying to load lz.vsgt.

[Robert Osfield]

On Wed, 25 May 2022 at 11:16, 'Denis Taniguchi' via vsg-users : VulkanSceneGraph Developer Discussion Group <vsg-...@googlegroups.com> wrote:

Oh, forgot to mention I am trying to load lz.vsgt.

Running:

   vsgdynamicload models/lz.vsgt

Crashes with seg fault for me.  This is an issue I'm aware of - it's due to the viewer's main View not being configured with the required depth sorted bin that the lz.vsgt with it's transparent billboarded trees requires.  This is the same issue we've discussed before but in the context of .gltf model that also required depth sorted bin.

If you load models and assign them to the viewer prior to calling viewer.compile() then the VIewer::compile() is able to check the assigned scene graphs for any DepthSorted nodes and then create the bins as required.  So in the lz.vsgt model it'll find the trees and see that they are decorated with a DepthSorted node and then create the appropriate bin type and number.

If there is no scene graph already assigned then the Viewer::compile() doesn't have anything to spur it into creating a bin.  Then the loader thread loads the problem model and merges it with the main scene graph and then on the next frame the RecordTraversal will try to assign the trees to depth sorted bin that doesn't exist.

There are two solutions to this - preallocate the required bins when you set up the Views for the Viewer, or to have the loaded thread check for the bins and then when merging with the main scene graph check for the presence of required bins in the Views assigned to the view and if they aren't there assign them.

Making sure both options work is on my TODO list for 1.0, and one I'm planning to tackle this week.

 

Once I've got a solution to this problem both lz.vsgt and the problem .gltf models will work, as will any other model with their own requirements of binning.

--

As a general note, the OSG took the approach of dynamically creating all the bins on the fly during each frame's cull traversal.  This was super flexible but it meant lots of checks to see if the required bins were available, if not then create them, then assign the rendering leafs to them.  Checks and creating structures on the fly is expensive so not great for performance.

With the VSG I've chosen the route of making assumption that the required resources/structures are set up at scene graph/viewer  creation time so that during the RecordTraversal there is no need for any checks - it just just look up the container and directly use it.  Lots of decisions like this is a big part of why the VSG scales much better than the OSG.

However, it does mean that some more housekeeping work is required to keep everything running smoothly.  If all goes to plan this particular pain point will be neatly handled by the end of this week.

[Robert Osfield]

Making sure both options work is on my TODO list for 1.0, and one I'm planning to tackle this week.

 

Once I've got a solution to this problem both lz.vsgt and the problem .gltf models will work, as will any other model with their own requirements of binning.

I created DynamicSceneGraphs branches of VulkanSceneGraph and vsgExamples to investigate and experiment with possible solutions for the crash in vsgdynamicload when you load a model that has transparent subgraphs that have a vsg::DepthSorted node.

Originally this would crash:

    vsgdynamicload models/teapot.vsgt models/lz.vsgt 

With the changes checked in the DynamicSceneGraphs branches this now runs safely.  Changes to VulkanSceneGraph are:

   https://github.com/vsg-dev/VulkanSceneGraph/commit/9b60ceb308996720f3b60f7048967cc4c315ff8e

Changes to vsgExamples are:

    https://github.com/vsg-dev/vsgExamples/commit/3be44613fe5d0d7d5eb214015fb1afeb3d45939e

What I'd also like to handle is when new subgraphs have PagedLOD so need to enable the DatabasePager and to handle when the maxSlot number increases. 

I also am thinking about integrating more of the functionality presently in vsg::DatabasePager and vsg::DynamicLoadAndCompile class from vsgdyanmicload directly into vsg::Viewer as it'd be good to avoid the complexity of internal scene graph/view house keeping required when handling dynamic scene graph from having to be handled by end users. 

I haven't figured out what the final changes will be, so consider the changes above as prototyping/proof of concept.  Broadly I'd like to make it easy for users to load or create a new scene graph and then compile and add it to the viewer with a couple of simple calls.  If I get this right then the DatabasePager and the DynamicLoadAndCompile classes should shrink with more of the functionality they currently have to handle being centralized.

Cheers,

Robert.

[Robert Osfield]

I have checked in an experiment with moving the scene graph compilation from the DynamicLoadAndCompile class into the viewer, as this is experimental I've created a CustomViewer class in the vsgdynamicloadexample and implemented a local compile function:

    void compile(vsg::ref_ptr<vsg::Object> object, vsg::ResourceRequirements& requirements);

This allows the DynamicLoadAndCompile class in vsgdyanmicload ti be simplified so that the load call can then immediately call the viewer->compile(loadedModel, requirements) rather than having a separate thread and compile functor to handle the compilation stage.  The thread safe list of CompileTraversal has also been moved from the DynamicLoadAndCompile class into the CustmViewer further simplifying it. 

I also updated the vsg::CompileTraversal to use a reference to Viewer rather than pass in a ref_ptr<Viewer> as this just makes more sense for the usage case, and makes clear that the CompileTraversal shouldn't ever take a ref_ptr<> to a Viewer is the Viewer now owns the CompileTraversal so avoiding any potential circular references.

Changes to vsg::CompileTraversal:

   https://github.com/vsg-dev/VulkanSceneGraph/commit/612fed0cf315eba0a271e7a68fe94664e326bfdd

Changes to vsgExamples vsgdyanmicload:

   https://github.com/vsg-dev/vsgExamples/commit/55b5376e41d1d72dd46e8e7e9c2879524ebd15f9

This afternoon I'm going to look at how to generalize the merge operation so that too can be handled more directly by the Viewer class rather than the DynamicLoadAndCompile class.

I don't know if I'll be able to get to this point, but it might be possible to do away with the DynamicLoadAndCompile class completely and just have a std::thread load the models and call viewer.compile() and viewer.addUpdateOperaton(mergeOperation).

Cheers,

Robert.

[Robert Osfield]

 > This afternoon I'm going to look at how to generalize the merge operation so that too can be handled more directly by the Viewer class rather than the DynamicLoadAndCompile class.

I have moved the merge operation from DynamicLoadAndCompile class into a Merge functor that gets called by the CustomViewer::update().  This cleans up the DynamicLoadAndCompile class and the application main loop.  Changes are:

    https://github.com/vsg-dev/vsgExamples/commit/f111c5586c2ff4881c4762155a83dc27a644b23a

This leaves the DynamicLoadAndCompile class just providing 1/3rd of the functionality that it once, while the other 2/3rd is still hanging around the vsgdynamicload example this is only temporary while experiment with the how to manage this functionality, once I'm confident that this approach is best approach I'll merge the CustomViewer functionality into vsg::Viewer.

Once this new functionality is merged with vsg::Viewer it should also be possible to simplify the vsg::DatabasePager if it too will no longer be required to explicitly handle the compile and merge operations.

I will now see if I can replace the DynamicLoadAndCompile class entirely by using either vsg::OperationThread/OpertionQueue, or a std::thread.  I'm aiming to distill down the code required to implement multi-threaded database loading while encapsulating the more complex tasks to make it easier to use and more robust.

[Robert Osfield]

> I will now see if I can replace the DynamicLoadAndCompile class entirely by using either vsg::OperationThread/OpertionQueue, or a std::thread.  I'm aiming to distill down the code required to implement multi-threaded database loading while encapsulating the more complex tasks to make it easier to use and more robust.

I've now completed the removal of the DynamicLoadAndCompile class, so now there are just a LoadOperation  and Merge classes that are used with a standard vsg::OpenThreads which is setup with 12 threads

    https://github.com/vsg-dev/vsgExamples/commit/ebe022716713f36d1ff82e1c4ceee92e332abc6f

The vsgdynamicload.cpp still contains the local CustomViewer class that I will removed once the functionality it provides is moved into the core vsg::Viewer class.

I'm now have got the code into the right ballpark of where I wanted it to be, enough of the ideal of refactoring vsg::Viewer and the vsgdynamicload examples has been been fleshed out to know that it's a workable and pretty clean approach.  I haven't yet fleshed out all the design/implementation details so let the current prototyping work rest overnight and I'll return tomorrow to start fleshing out the final design and implementation.

One issues I need to flesh out are how to handle new View's being added, how to pass on a new max slot number increasing and how to enable the DatabasePager if a new sub-graph requires it, and what happens if a background thread calls viewer->compile(ref_ptr<Node>) before the main viewer->compile() is called.

I also want to simplify the work required to merge new subgraphs and update the viewer in appropriate ways if required.

OK. Perhaps I'm not that close to finishing this work after all...

[Robert Osfield]

I may have been quiet on this thread for a few days, but have been improving the support for sharing vsg::MemoryBufferPool, vsg::Buffer, vsg::CommandPool/CommandBuffer etc. between threads so concurrent allocation/freeing of resources can happy safely.  For the most part it won't affect most VSG users as these affect classes used by the VSG backend.  The changes to the VSG are:

   https://github.com/vsg-dev/VulkanSceneGraph/commit/428ed113235bf34add24a44a0416ca93d6969696

However, if you use the submitCommandsToQueue(..) parameter has changed a bit so if you use this function then you'll need to update your code.  The initial Deivce* device parameter is no longer required.  It now looks like:

    template<typename F>
    void submitCommandsToQueue(CommandPool* commandPool, Fence* fence, uint64_t timeout, Queue* queue, F function)

The changes checked into vsgExamples include updates for this change are:

   https://github.com/vsg-dev/vsgExamples/commit/157482302b2a10b11ee372f7b5d381df82fb271b

The changes to vsgcompute and vsgscreenshot are fixes for the  submitCommandsToQueue() change.  Most of the changes are to vsgdynamicload example which now leverages more core VSG classes - made possible by refinements like the introduction of a vsg::TheadSafeQueue template class.

The vsgdynamicload example still has the CustomViewer class that extends the Viewer class to add compile support.  I haven't yet completed all the functionality so alas I'll need to return to this work next week.  Once I'm happy I'll merge the CustomViewer functionality into vsg::Viewer and vsgdynamicload can finally shrink down and be a lot simpler for users to follow and adopt for their own code.

Cheers,

Robert.

[Robert Osfield]

I have now added support for changes to CommandGraph::maxSlot number onto the vsg::State::stateStacks size so that applications can change the CommandGraph's maxSlot during the life of the application and have it automatically passed on.  The change for this is delightfully simple (checked into the VSG DynamicSceneGraphs branch):

    https://github.com/vsg-dev/VulkanSceneGraph/commit/907fba5a9cc37821e55f9c800946576122bfa760

I have also implement proof of concept support for enabling vsg::DatabasePager :

    https://github.com/vsg-dev/vsgExamples/commit/c8a097e69065fb15a64b76e38315d0d70039e3d4

For testing I've run vsgdynamicload with all the 3d models in vsgExamples/data/models along with the FlightHelment model from glTF-Sample-Models:

$ vsgdynamicload lz.vsgt openstreetmap.vsgt raytracing_scene.vsgt readymap.vsgt teapot.vsgt ~/Data/glTF-Sample-Models/2.0/FlightHelmet/glTF/FlightHelmet.gltf  
CompileTraversal::add(window =0x7f22d04ae730, view = 0x7f22d03ad130)
Merge::run() path = lz.vsgt, ref_ptr<vsg::Group>(vsg::MatrixTransform 0x7f22d03ad3a8), ref_ptr<vsg::Node>(vsg::MatrixTransform 0x7f22d03b57c8)
Merge::run() path = openstreetmap.vsgt, ref_ptr<vsg::Group>(vsg::MatrixTransform 0x7f22d03ad468), ref_ptr<vsg::Node>(vsg::MatrixTransform 0x7f22d03b5d50)
Merge::run() path = raytracing_scene.vsgt, ref_ptr<vsg::Group>(vsg::MatrixTransform 0x7f22d03ad530), ref_ptr<vsg::Node>(vsg::MatrixTransform 0x7f22d03b66a0)
Merge::run() path = readymap.vsgt, ref_ptr<vsg::Group>(vsg::MatrixTransform 0x7f22d03ad608), ref_ptr<vsg::Node>(vsg::MatrixTransform 0x7f22d03b6ed8)
Merge::run() path = teapot.vsgt, ref_ptr<vsg::Group>(vsg::MatrixTransform 0x7f22d03ad6c0), ref_ptr<vsg::Node>(vsg::MatrixTransform 0x7f22d03b71f8)
Merge::run() path = /home/robert/Data/glTF-Sample-Models/2.0/FlightHelmet/glTF/FlightHelmet.gltf, ref_ptr<vsg::Group>(vsg::MatrixTransform 0x7f22d03ad7b8), ref_ptr<vsg::Node>(vsg::MatrixTransform 0x7f22d03b8040)
radv/amdgpu: Not enough memory for command submission.
Context::waitForCompletion()  0x7f22d04c13a0 fence->wait() failed with error. VkResult = -4 

It runs and loads up the all the models in background thread and then enables DatabasPaginging automatically, but alas it then runs out of memory on my AMD5700G.  I'm surprised by this so will need to tinker further.  It is loading two whole earth paged databases but I wouldn't have thought that would be a major issue as I haven't zoomed in to load high rest tiles.

This is what I see:

I would like to add support for adding new views as this another aspect to dynamic scene graph / viewer that users will throw at the VSG, so I'll add a test into the vsgdynamicload for this case too - perhaps load a model then create a new View for it in the background then, then in the merge operation update the CompileTraversals to handle the new view, and assign it to the appropriate RenderGraph.

Cheers,

Robert.

[Robert Osfield]

Hi All,

The next step has been to create vsg::CompileManager to handle the compile of subgraphs in a thread safe way, this is now checked into the DynamicSceneGraph branch of the VSG:

     https://github.com/vsg-dev/VulkanSceneGraph/commit/88f5502f28521fb3178d2d513eb51b64d30acca7

This addition means we can simplify the vsgdynamicload example substantially, now there is now CustomViewer class - just Merge and Load Operation classes to do the work and a help function for updating viewer related state.  This helper function will eventually be moved into the core VSG in some fashion.

    https://github.com/vsg-dev/vsgExamples/commit/bdbedb298d0a15a780043e5d0b818e5d7dac7de4

The CompileManager also enables the DatabasePager to be cleaned up, removing the local compile thread and associated containers, this saves 158 lines of code:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/0fdddfb4d74ccf2905f8afd47b7fb58342438b6a

The use of CompileManager in both the multi-threaded load/compile in vsgdynamicload and in the DatabasePager resolves the issues as I was see a few days ago when loading paged datatabases in vsgdynamicload.

Cheers,

Robert.

[Robert Osfield]

I spoke too soon on paged databases now working well in vsgdynamicload, it looks like the scaling of the models to fit alongside each other is breaking the LOD selection in some way so that higher and higher res tiles are requested. Eventually my system throws errors after 6723 of active requests have piled up:

DatabasePager::updateSceneGraph() numActiveRequests = 6720
DatabasePager::updateSceneGraph() numActiveRequests = 6723

radv/amdgpu: Not enough memory for command submission.

VUID-vkQueuePresentKHR-pWaitSemaphores-03268(ERROR / SPEC): msgNum: 622825338 - Validation Error: [ VUID-vkQueuePresentKHR-pWaitSemaphores-03268 ] Object 0: handle = 0x55d779911750, type = VK_OBJECT_TYPE_QUEUE; Object 1:
handle = 0xfa21a40000000003, type = VK_OBJECT_TYPE_SEMAPHORE; | MessageID = 0x251f8f7a | vkQueuePresentKHR: Queue VkQueue 0x55d779911750[] is waiting on pWaitSemaphores[0] (VkSemaphore 0xfa21a40000000003[]) that has no wa
y to be signaled. The Vulkan spec states: All elements of the pWaitSemaphores member of pPresentInfo must reference a semaphore signal operation that has been submitted for execution and any semaphore signal operations on
which it depends (if any) must have also been submitted for execution (https://vulkan.lunarg.com/doc/view/1.3.211.0/linux/1.3-extensions/vkspec.html#VUID-vkQueuePresentKHR-pWaitSemaphores-03268)
   Objects: 2
       [0] 0x55d779911750, type: 4, name: NULL
       [1] 0xfa21a40000000003, type: 5, name: NULL
VUID-vkAcquireNextImageKHR-swapchain-01802(ERROR / SPEC): msgNum: 1050126472 - Validation Error: [ VUID-vkAcquireNextImageKHR-swapchain-01802 ] Object 0: handle = 0xdd3a8a0000000015, type = VK_OBJECT_TYPE_SWAPCHAIN_KHR; |
MessageID = 0x3e97a888 | vkAcquireNextImageKHR: Application has already previously acquired 1 image from swapchain. Only 1 is available to be acquired using a timeout of UINT64_MAX (given the swapchain has 3, and VkSurfa
ceCapabilitiesKHR::minImageCount is 3). The Vulkan spec states: If the number of currently acquired images is greater than the difference between the number of images in swapchain and the value of VkSurfaceCapabilitiesKHR
::minImageCount as returned by a call to vkGetPhysicalDeviceSurfaceCapabilities2KHR with the surface used to create swapchain, timeout must not be UINT64_MAX (https://vulkan.lunarg.com/doc/view/1.3.211.0/linux/1.3-extensi
ons/vkspec.html#VUID-vkAcquireNextImageKHR-swapchain-01802)
   Objects: 1

[Robert Osfield]

On Wed, 8 Jun 2022 at 10:26, Robert Osfield <robert....@gmail.com> wrote:

I spoke too soon on paged databases now working well in vsgdynamicload, it looks like the scaling of the models to fit alongside each other is breaking the LOD selection in some way so that higher and higher res tiles are requested. Eventually my system throws errors after 6723 of active requests have piled up

I should add this LOD selection issue when scaling a paged database from the diameter of the earth down to a unit scale really isn't a bug with threading or compilation with my current development goal, so I'll generate an Issue on githib for it and then return to the main task in hand  - wrapping up the dynamic scene graph support.  The remaining item is adding support for new Views.

[Robert Osfield]

On Wed, 8 Jun 2022 at 10:30, Robert Osfield <robert....@gmail.com> wrote:

 The remaining item is adding support for new Views.

I've now completed work on adding support for dynamically adding new vsg::View, with the new vsg::CompileManager providing both the public API to support it, and the internal house keeping to keep things in order in a thread safe way.  I have updated the vsgdyanmicload example to illustrate how it's managed, the relevant code is ([padded out with debug console output) :

           // need to add view to compileManager
            ref_viewer->compileManager->add(window, view);

            auto result = ref_viewer->compileManager->compile(renderGraph, [&view](vsg::Context& context)
            {
                if (context.view == view.get())
                {
                    std::cout<<"    Enabling compile for view "<<view<<std::endl;
                    return true;
                }
                else
                {
                    std::cout<<"    Disabling compile for view "<<view<<std::endl;
                    return false;
                }
            });

The two parts are adding the view to the compileManager so that future scene graph compilation can compile for it, and then use of lambda function that provides a ContextSelectionFunction to select from the vsg::Context used to compile each of the device/views that are managed by the CullTraversals held in the CompileManager.

With the latest work the DynamciSceneGraph branch work is now feature complete.  For the rest of this morning I'll do code reviews and tighten up the API and implementation and will merge with VSG and vsgExamples master this afternoon.

Changes to VulkanSceneGraph are:

    https://github.com/vsg-dev/VulkanSceneGraph/compare/DynamicSceneGraphs

Changes to vsgExamples are:

    https://github.com/vsg-dev/vsgExamples/compare/DynamicSceneGraphs

Now would be a good time for the community to check out the DynamicSceneGraph branches and let me know of any problems/make suggestions so I can make any required revisions before I merge with the respective master.

Cheers,

Robert.

[Robert Osfield]

I have moved the updateViewer(Viewer& viewer, CompileResult& compileResult) function from the vsgdynamicload example into the core VSG:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/9d806ac07b3ecc59e103e4fe2251cbcb83eb6739

This function updates the vsg::RecordAndSubmitTasks for any new maxSlot values, requirements for vsg::DatabasePager and to allocate an vsg::Bin.  Most of this functionality is implemented by updateTasks(RecordAndSubmitTasks, CompileResult) function, I've done this in case users want to implement their own viewer classes.

I updated vsgdynamicload to reflect the move of the function into the core VSG, and have moved the dynamic View operations into a new vsgdynamicviews example.  This slims down the vsgdynamicload example and brings it in line with it's original features before the DynamicSceneGraphs branch work.  Changes to vsgExamples:

    https://github.com/vsg-dev/vsgExamples/commit/9d287fc7d4407d6d43fe1915fa2dd9f7c0687184

With these  improvements to the core VSG the vsgdynamicload example has shrunk in size from 339 lines to 216 and it works more robustly, handling paged databases, increases in slot numbers and new Bin requirements that the old example could handle.

This set of check-in wraps up the functionality side, after lunch I'll do a core review and all going well merge with VSG and vsgExample master.

[Robert Osfield]

Hi All,

I've now done all my changes and code reviews, everything looks good so I've merged the DynamicSceneGraph branches of VSG, vsgImGui and vsgExamples with their respective masters:

Changes to VulkanSceneGraph:

    https://github.com/vsg-dev/VulkanSceneGraph/pull/458

Changes to vsgImGui (includes update to ImGui 1.87):

    https://github.com/vsg-dev/vsgImGui/pull/18

Changes to vsgExamples:

    https://github.com/vsg-dev/vsgExamples/pull/153

With these merges I can tick off another major refinement items that I had as a MUST have for VulkanSceneGraph-1.0.  I can also close some long standing github Issues :-)

Cheers,

Robert.

[Robert Osfield]

To illustrate vsgdynamicload in action here's a screenshot of it having loaded models from the official glTF dataset, the unix command line I used to do this was:

    find . -name "*.gltf" | head -n 100 | xargs vsgdynamicload

[Robert Osfield]

This is a screenshot of the new vsgdynamicviews example - it's just loading vsgExamples/data/model datasets, the main view of the teapot is loaded in the main loop, then the two Views of the earth and lz models are loaded in a background thread, decorated with a Rendergraph/View and assigned to the viewer via the new viewer.addupdateOperation() feature (vsgdynamicload now uses this as well.)

As well as adding the Views in a multi-threaded way, this particular model combination illustrates the loading of paged database with the need for a vsg::DatabasePager being detected, then one is automatically created and assigned to the viewer's RecordAndSubmitTasks so straight away it begins paging in data.  The second view illustrates the automatic detection of transparent depth sorted trees which require a vsg::Bin to be created and assigned to the View.  Again this is done automatically for you - this is done within the new vsg::updateViewer(..) function.