vsg::Camera and vsg::visit<> developments

[Robert Osfield]

Hi All,

This week I've experimented with making it easier to provide camera settings via scene graph, rather than solely at the application/viewer level.  The aim of this work to make it possible to load a scene graph, such as from the vsgXchange/Assimp loader that have camera settings provided in them as nodes, and then utilize these when setting a viewer level cameras.

To achieve this functionality I've changed vsg::Camera so that it now subclass from vsg::Node rather than vsg::Object.  This enables users to just add a vsg::Camera directly as child in the scene graph.  I've also added serialize support so you can read/write them as part of the scene graph.  I have also added a std::string name member variable to vsg::Camera to allow you to give a string name to cameras so they can be later located in searches.

To help locate the cameras in the scene graph I have added a vsg::FindCameras visitor, and to streamline the creation and visitation of the scene graph I've add new vsg::visit<> templated function so to find all the cameras and then print out where in the scene graph they were found you just need to do:

   auto scene_cameras = vsg::visit<vsg::FindCameras>(scenegraph).cameras;
    for(auto& [nodePath, camera] : scene_cameras)
    {
        std::cout<<"\ncamera = "<<camera<<", "<<camera->name<<" :";
        for(auto& node : nodePath) std::cout<<" "<<node;
        std::cout<<std::endl;
    }

The vsg::visit<> template function is something that just popped up as an idea for streamlining the Visitor usage, but isn't limited to camera related functionality, it should be useable in number of places for making visitor usage more expressive and streamlined.

I have also added support for mapping Assimp's aiCamera objects to equivalent vsg::Camera objects placing them under the appropriate transforms in the scene graph. To illustrate this all in action I've also created a new vsgcameras example, which using the above code and loading file via vsgXchange/Assimp we see on the console:

$ vsgcameras ./FBX/global_settings.fbx

camera = ref_ptr<vsg::Camera>(vsg::Camera 0x55a2d1335d40), Camera.001 : ref_ptr<vsg::Object>(vsg::MatrixTransform 0x5
5a2d133d910) ref_ptr<vsg::Object>(vsg::StateGroup 0x55a2d1335c80) ref_ptr<vsg::Object>(vsg::MatrixTransform 0x55a2d12
bfc20) ref_ptr<vsg::Object>(vsg::MatrixTransform 0x55a2d13503e0) ref_ptr<vsg::Object>(vsg::Camera 0x55a2d1335d40)

camera = ref_ptr<vsg::Camera>(vsg::Camera 0x55a2d1335de0), Camera : ref_ptr<vsg::Object>(vsg::MatrixTransform 0x55a2d
133d910) ref_ptr<vsg::Object>(vsg::StateGroup 0x55a2d1335c80) ref_ptr<vsg::Object>(vsg::MatrixTransform 0x55a2d12bfc2
0) ref_ptr<vsg::Object>(vsg::MatrixTransform 0x55a2d12bfce0) ref_ptr<vsg::Object>(vsg::Camera 0x55a2d1335de0)
Using a single RenderGraph, with both Views separated by a ClearAttachemnts

The new vsgcamears also also creates an event handler that maps a viewer's View's Camera's ViewMatrix to the location of the of in scene graph cameras in response to 1 2 etc. key presses.  Pressing 0 returns to the default viewing position.

To help with tracking moving cameras in the scene graph I've add a vsg::ViewTrackObject ViewMatrix subclass so you use the nodePath that locates an in scene graph camera in conjunction with vsg::computeTransform(nodePath)so the View will automatically be updated on each frame - so there is no need to manually updated.  This can be used from having a View that tracks a camera on a robot located in scene and moving around.

All these changes can be found in the CameraNode branches of VulkanSceneGraph, vsgXchange and vsgExamples:

    https://github.com/vsg-dev/VulkanSceneGraph/tree/CameraNode

    https://github.com/vsg-dev/vsgXchange/tree/CameraNode

    https://github.com/vsg-dev/vsgExamples/tree/CameraNode

The vsgcameras examples doesn't yet doesn't yet demonstrate all the functionality I want it to, at the core VSG and vsgXchange level I think we're now mostly there.  The additions to vsgcameras I am planning to use of the vsg::ViewTrackObject to bind View's to track objects and cameras in the scene graph.

I am open to suggestions of better names for vsg::ViewTrackObject and enhancements with the vsg::visit<> template function, vsg::Camera, so let me know what you think.  

Cheers,

Robert.

[Robert Osfield]

Hi All,

Today I've worked on refining the new vsg::Camera related features and extending the new vsgcameras example to illustrate several of the new features.  A picture is worth a thousand words so here's progress so far:

     vsgcameras glTF-Sample-Models/2.0/FlightHelmet/glTF/FlightHelmet.gltf

The glTF-Sample-Models used here is the official glTF examples hosted on github:

    https://github.com/KhronosGroup/glTF-Sample-Models

As part of the work today I decided to change the name of recently added vsg::ViewTrackObject to vsg::TrackingViewMatrix as it rolls better off the tongue and hopefully is more obvious as to it's function.  I also renamed vsg::RelativeView to vsg::RelativeViewMatrix, in this case to make it clearer it's a subclass of ViewMatrix, rather than a vsg::View.

The vsgcameras example uses the new vsg:::FindCameras visitor to locate all the cameras stored in the scene graph and then creates a separate vsg::View for each one placing them as column on the right right side of the window.  If the loaded scene graph doesn't contain any in scene graph cameras then the example creates a Top, Left, right and Front camera, placing them in the scene graph then re-runs the FindCameras so that the subsequent View set up code as something to work with.

When running the vsgcamras example pressing the key 1 through to the maximum number of in scene cameras, 4 in the above instance, sets the main View's LookAt to the LooAt of the selected scene camera.    Pressing ' ' returns to the default camera view set up during initialization.

The Views down the side use the vsg::TrackingViewMatrix to track the in the associated scene camera, which will accumulate any transforms above that in scene camera, this should enable the View to track moving objects in the scene.  The example doesn't generate any animation transforms for this yet, but that's something for a further iteration, though perhaps this should be left to another example as this one is already jam packed with features it's illustrating.

[Robert Osfield]

Hi All,

I have now merged the CameraNode branches of the VSG, vsgXchange and vsgExamples with their respective masters.

Cheers,

Robert.

[vahid shahvar]

Hi Robert,

I do not get the correct result from codes vsgcameras and vsgmultiviews. In code vsgmultiviews, when I set seperateRenderGraph = false, the result is correct, but not with seperateRenderGraph = true.

I attached an image for the vsgcameras output.

regards,

Vahid

[Robert Osfield]

HI Vahid,

On Wed, 25 May 2022 at 10:58, vahid shahvar <vahid....@gmail.com> wrote:

I do not get the correct result from codes vsgcameras and vsgmultiviews. In code vsgmultiviews, when I set seperateRenderGraph = false, the result is correct, but not with seperateRenderGraph = true.

I attached an image for the vsgcameras output.

regards,

I've just run vsgcameras and vsgmultiviews models/lz.vsgt on my Kubuntu 21.10 system with AMD graphics and everything looks OK.  Do they look OK for you in the default configuration like this?

What OS, hardware and VSG version are you using?

Could you run the applications with -d on the command line to enable the Vulkan debug layer to see if there are any issues.

 

[vahid shahvar]

Hi Robert,
Thanks for your response.

I think the problem is using the separate Render Graph (Using a RenderGraph per View).

When I run vsgmultiviews with default setting models/lz.vsgt, the result is Ok. But with models/lz.vsgt -s l just see secondary_view.

Also I don’t get correct output from vsgcameras with models/lz.vsgt, because it uses also two Render Graph.

My system config is:
Windows 10, Nvidia Quadro M2200, VSG 0.2.10
Microsoft visual studio Professional 2019.

In addition l didn’t see any issues by enabling -d on command line.

Regards,
Vahid

[Robert Osfield]

Hi Vahid,

I have just tried:

$ vsgmultiviews models/lz.vsgt -d
Using a single RenderGraph, with both Views separated by a ClearAttachments

And

$ vsgmultiviews models/lz.vsgt -d -s
Using a RenderGraph per View

Both produce identical results on my Kubuntu 21.10 + AMD 5700G system, both views are visible and trackball works in both.

I also tried vsgcameras and it correctly displays the main and 4 insert views.

 $vsgcameras models/lz.vsgt -d

The is the VulkanSceneGraph and vsgExamples master. As there are no debug errors being reported in any configuration there isn't any obvious thing to look into.

Do you have any other systems that you could try things on?

Could others in the community try out the above command lines and report back what they see?

Cheers,

Robert.

[vahid shahvar]

Thanks Robert,

I haven't gotten these results with the same command line settings.

I will try to test it on another system later, but for now I only have this system.

And it would be very helpful if others in the community could help with this test.

Regards,

Vahid

[Rainer Gericke]

Hi Robert and All,

I am using a Win10 Laptop with NVIDiA Geforce RTX 3080 graphic card and Visual Studio 2022. I got the same results as you showed.

vsgmultiviews models/lz.vsgt -d

vsgmultiviews models/lz.vsgt -d -s

vsgcameras models/lz.vsgt -d

Best

[Gao ZhiHua]

On Windows 10 with NVIDIA Quadro P620,  With -s option,  I can only see the upper corner view, alone with the black main view.   Even with -d -s,  I can't see any other info from cmd window.

[Robert Osfield]

On Sun, 27 Nov 2022 at 06:41, Gao ZhiHua <crazyr...@gmail.com> wrote:

On Windows 10 with NVIDIA Quadro P620,  With -s option,  I can only see the upper corner view, alone with the black main view.   Even with -d -s,  I can't see any other info from cmd window.

Could you try this example on other OS/hardware?

Which version of the VSG are you using?

Do any of the other vsgExamples look like they might not be working?

[Gao ZhiHua]

Hi Robert,

Thanks a lot for your response.
As far as I know, all other examples work fine except this one.

I tested this example on another computer, Windows 11 with NVIDIA RTX 3060, it works fine with -s option.
I pulled source code after vsg 1.0 was released by a script, including source code of VulkanSceneGraph,  osg2vsg, vsgxchange, vsgGIS, vsgExamples. In fact, I noticed this problem several months ago on the computer with NVIDIA Quadro P620 but think I could wait for new release.

This is the git log output under VulkanSceneGraph repo:

➜ VulkanSceneGraph git(master) git log
commit cb054a90960da565cd5e39a465534fd37fd3b413 (HEAD -> master, origin/master, origin/HEAD)
Merge: fbe8e499 17fc3ef0
Author: Robert Osfield <rob...@openscenegraph.com>
Date:   Tue Nov 15 18:46:43 2022 +0000

    Merge pull request #585 from sbrkopac/cygwin-fix

    Package workaround for egor-tensin/setup-cygwin#9

commit 17fc3ef09e430b0243da31bcc6c32b94c5c1b81d
Author: Sam Brkopac <sbrk...@users.noreply.github.com>
Date:   Tue Nov 15 11:32:47 2022 -0700

    workaround for broken github action

......

Regards
Eric

[Robert Osfield]

Hi Eric,

On Mon, 28 Nov 2022 at 02:32, Gao ZhiHua <crazyr...@gmail.com> wrote:Hi Robert,

Thanks a lot for your response.
As far as I know, all other examples work fine except this one.

I tested this example on another computer, Windows 11 with NVIDIA RTX 3060, it works fine with -s option.
I pulled source code after vsg 1.0 was released by a script, including source code of VulkanSceneGraph,  osg2vsg, vsgxchange, vsgGIS, vsgExamples. In fact, I noticed this problem several months ago on the computer with NVIDIA Quadro P620 but think I could wait for new release.

Unfortunately I don't have a system that shows this problem and as there are no debug layer warnings I don't have any leads into what might be amiss.  It could be a driver issue, it could be a bug lurking in the example or in the core VSG, there is no way to know at this point.

The best I can recommend is trying out the same example on hardware and OS's to see if a pattern emerges for where things work and where things don't.

FYI, vsgGIS functionality has been merged into vsgXchange so you no longer need vsgGIS.

Cheers,

Robert.

[Gao ZhiHua]

Hi Robert,

Thanks a lot for your advice, I will try out, and if I could find something useful, will post it here.

Regards,

Eric

[Gao ZhiHua]

Hi Robert,

Captured by renderdoc,  I got this:

It looks like, when renderArea is set,  vkCmdBeginRenderPass (C=Clear, D=Clear) makes content outside render area to be undefined.

I captured this in the same way on the computer with RTX 3060,  renderdoc shows same undefined content, but the vsgmultiviews gives correct display result surprisingly.

I got confused totally ..........

[Robert Osfield]

From these findings it sounds like an issue with a missing scissor test. I will look into this once I get back to my main dev system. 

[Robert Osfield]

On Sun, 4 Dec 2022 at 08:32, Robert Osfield <robert....@gmail.com> wrote:

From these findings it sounds like an issue with a missing scissor test. I will look into this once I get back to my main dev system. 

I have investigated the setup of the vkBeginRenderPass in vsg::RenderGraph and everything looks correct with the renderArea.  The RenderPass settings look correct to me as well.

The GraphicsPipeline setup that uses the ViewportState also looks to be applying the right values.  The CompileTraversal/Context setup looks to be a bit over engineered but all the correct values look to be passed on when required.

I don't know what else I can look at on the VSG side. My best guess at this point is that the VSG is doing the right thing, with the Vulkan driver mishandling what is otherwise correct Vulkan calls/settings. 

We are now getting to a point where we need to approach NVidia or the wider Vulkan community about the problem.

Cheers,

Robert.