Please provide feedback : Tasks left to complete VulkanSceneGraph-1.0

[Robert Osfield]

Happy New Year folks :-)

I'm planning out my next few months of work on the VulkanSceneGraph to prepare for it's first stable release, with an aim to make the release this Spring.  I have tasks I feel are necessary before 1.0 and plan to undertake myself, I'll enumerate these below, but I'd like to know what the community feel be worthwhile tackling prior to 1.0.

I've updated the VulkanSceneGraph/ROADMAP.md with the timeline and may items I have on my TODO list.  The part that is relevant to this final phase is:

Pending/Underway tasks:

• Positional state support to enable easier support of lighting, shadows, texture projection.
• Support for integration with OpenGL/OSG applications via EXT_external_object & VK_KHR_external_memory
• Port to iOS

4. Release Phase, Spring 2022

Goal: Test scene graph library against real-world applications and shake down the API and implementation for it's first stable release.

• Refinement of API and implementation
• Build relationships with application developers and involve them in testing
• Create tutorial and example programs to illustrate how to use VSG
• Test, debug, refine and release 1.0.0!

--

The next major item that I'll be tackling this month is the item "Positional state support to enable easier support of lighting, shadows, texture projection". I have already begun provisional work fleshing out the breadth of the work and what classes would be necessary. 

My aim will be to provide general purpose pipeline configuration scheme that fully support PBR, lighting, basic shadows and texture projections, with elements like vsg::Builder and the OSG and Assimp loaders in vsgXchange ported across to all use it.  If it works out how I'd like it to this will give us a unified set of default approach to state and shaders to use, and a scheme that allows users to customize it for their own needs.

It's pretty ambitious work so while I've pencilled this in for this month it's likely to roll over into subsequent months, and be likely something that would continue to evolve even after 1.0 is made.

This work will directly impact vsgXchange, requiring internals of the OSG and Assimp loaders to be rewritten.  This will also open the door to addressing some of the bugs/limitations in the present OSG loader, so rather than try to fix these OSG loader problems first, I'll do the general purpose work first, then port the OSG loader across and address the problems then.

I will start up a separate thread for this work so I'd ask you to keep you comments/suggestions on this work to that thread.

--

When posting suggestions here about issues or new features you'd like tackled before VulkanSceneGraph-1.0 can you provide some input on the priority you see it.  I find the broad priority "MUST HAVE," "Should have", "could have" demarcation useful when discussing priorities. 

One typically tackles the "MUST HAVE" items first, then you assess the timeline to see what "Should have" or even "could have" items could be tackled for the 1.0 releases.

When making suggestions please also let us know what assistance, in manpower or funding, you can provide in developing and testing the features.  This applies not only to your own preferred items but also ones that others a proposing.  The more we can help each other the quicker we'll get across the 1.0 finishing line.

Thanks for you input and help over the next few months,

Robert.

[Robert Osfield]

On Monday, 3 January 2022 at 11:19:31 UTC Robert Osfield wrote:

The next major item that I'll be tackling this month is the item "Positional state support to enable easier support of lighting, shadows, texture projection". I have already begun provisional work fleshing out the breadth of the work and what classes would be necessary. 

...

I will start up a separate thread for this work so I'd ask you to keep you comments/suggestions on this work to that thread.

I have now created this thread: Extensible State/Pipeline Configuration with PBR, Lighting & Shadows
 

[Roland Hill]

Hi Robert,

This all looks great and it's exciting to see VSG moving towards version 1.0. The main feature I'm looking for is improvements to dynamically adding and removing subgraphs. I have a solution based on your DynamicLoadAndCompile example (PR https://github.com/vsg-dev/VulkanSceneGraph/pull/369), but you thought there were better ways to go about it.

Regards,

Roland

--
You received this message because you are subscribed to the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vsg-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/fcbc5f7b-c48e-48ba-bd1e-decbeae59a9dn%40googlegroups.com.

[Robert Osfield]

Hi Roland,

On Monday, 3 January 2022 at 13:24:31 UTC role...@gmail.com wrote:

This all looks great and it's exciting to see VSG moving towards version 1.0. The main feature I'm looking for is improvements to dynamically adding and removing subgraphs. I have a solution based on your DynamicLoadAndCompile example (PR https://github.com/vsg-dev/VulkanSceneGraph/pull/369), but you thought there were better ways to go about it.

I have left your PR open as a reminder that needs further work.

It would be worth crafting a work item for dynamic updates of the scene graph topic - there a number of different aspects to it.  I haven't experimented with the githib projects before but perhaps it's time to learn about how to use it as it would be useful to have a more formal way of laying out work plans and following progress on it.

Does anyone else have experience with using github projects?

Cheers,

Robert.

 

[Vadim V. Balashoff]

On Mon, Jan 03, 2022 at 03:19:31AM -0800, Robert Osfield wrote:

> ...but I'd like to know what the community feel be worthwhile tackling prior to 1.0.

I'm total noob in vulkan and opengl, so please forgive me if following is a
total shit.

1. examples, examples and more examples.
I think https://github.com/urho3d/Urho3D is best case of it. There are dozens
of simple sample projects in it. You can look exactly what you need.
So, for vsg samples MUST HAVE. Let it be couple of samples, but they have to
be compilable and running for current vsg.

2. widgets.
I'm not sure -- is all features from osg now implemented in vsg?
I can't find widgets.
I want to create ECS engine for vsg with ideas from this:
https://github.com/nmattia/Minotaur
but I want to get rid of CEGUI and create own UI based on native widgets.
More than this, I want to implement Polyphony ECS GUI ideas in it
(https://hal.inria.fr/hal-02147180/document)
There are some unusual ideas about ECS-way of input devices, and I like it.
So, there are widgets on osg, but are they on vsg?

3. editor.
I think without editor project is shuffling. For simple use of it there HAVE
TO BE an editor. Simplest one, but it have to display scene, allow to place
objects and particles and in best case have to play it in realtime just like
UE editor does.
again Urho3d editor just for info:
https://www.youtube.com/watch?v=11QDmTOkq38

4. voxels.
May be I'm wrong, but.
osg and vsg are positioning as "true earth modelling engines" and "if you
want more -- you are on your own. If it can it -- you are lucky. If not --
your problems".
I want voxels.
Yes, there are a lot of voxels engine for vulkan
(for example this one -- https://github.com/ericl85/vkvoxel)
but I'm simply stunned about this one:
https://www.youtube.com/watch?v=8ptH79R53c0
I'm not exactly sure about what I want exactly from vsg here, but if there
will be easy way to create voxel engine, it'll be great.
Even if it'll be STRAIGHT voxel engine, where ALL voxels are aligned on axis
without rotation possibilities.
(just like voxatron -- https://www.youtube.com/watch?v=ZWsUxlnNCbc)

v.v.b.

[Robert Osfield]

HI Vadim,

1. Examples : have you looked at the vsgExamples set?

       https://github.com/vsg-dev/vsgExamples

    If there are specific topics that aren't yet illustrated let us know.  For 1.0 the vsgExamples

    set will serve as a both a testbed of core VulkanSceneGraph features as they get developed

    and as examples for others to learn from.

2. Widgets : I don't think widgets are appropriate for the core VulkanSceneGraph, so won't

   be part of VSG-1.0.  However, if the community want/need an alternative to vsgImGui then

   I don't have any objection to them going ahead and writing one.

3. Editor : I can see value in this, though it'd be as an sibling project, rather than part of the

   VulkanSceneGraph itself.  Again this could be a community effort.

4. Voxels : I see domain specific techniques like this being sibling projects rather than part of

    the VulkanSceneGraph.  The core project is focused on general purpose scene graph that

    could be used to implement a range of different rendering or even compute applications.

While items 2, 3 & 4 aren't what I see as core to the VulkanSceneGraph-1.0 release I would like to make sure these are all possible using 1.0 as a base, so if there are design/implementation issues that prevent them from being developed then we need to spot what these are and address them prior to the 1.0 release.

Cheers,

Robert.

[Robert Osfield]

Hi All,

I have added the following to the ROADMAP.md:

• Improved support for dynamic scene graphs.
• Support for wide and standard strings in vsg::Path.

When I get to tackling these I'll need community input and testing, so if you have a particular usage case that touches upon these let me know.

Cheers,

Robert.

["François Cami"]

Happy New Year everyone :)

On Mon, Jan 3, 2022 at 12:19 PM Robert Osfield <robert....@gmail.com> wrote:
>
> Happy New Year folks :-)
>
> I'm planning out my next few months of work on the VulkanSceneGraph to prepare for it's first stable release, with an aim to make the release this Spring. I have tasks I feel are necessary before 1.0 and plan to undertake myself, I'll enumerate these below, but I'd like to know what the community feel be worthwhile tackling prior to 1.0.
>
> I've updated the VulkanSceneGraph/ROADMAP.md with the timeline and may items I have on my TODO list. The part that is relevant to this final phase is:
>
> Pending/Underway tasks:
>
> Positional state support to enable easier support of lighting, shadows, texture projection.
> Support for integration with OpenGL/OSG applications via EXT_external_object & VK_KHR_external_memory
> Port to iOS
>
> 4. Release Phase, Spring 2022
>
> Goal: Test scene graph library against real-world applications and shake down the API and implementation for it's first stable release.
>
> Refinement of API and implementation
> Build relationships with application developers and involve them in testing
> Create tutorial and example programs to illustrate how to use VSG

I'll happily proofread and peruse the tutorial. I'm a Vulkan noob as
it stands, but I'll get up to speed soon.

> Test, debug, refine and release 1.0.0!

As others have said before, this looks great!

Cheers
François

> --
>
> The next major item that I'll be tackling this month is the item "Positional state support to enable easier support of lighting, shadows, texture projection". I have already begun provisional work fleshing out the breadth of the work and what classes would be necessary.
>
> My aim will be to provide general purpose pipeline configuration scheme that fully support PBR, lighting, basic shadows and texture projections, with elements like vsg::Builder and the OSG and Assimp loaders in vsgXchange ported across to all use it. If it works out how I'd like it to this will give us a unified set of default approach to state and shaders to use, and a scheme that allows users to customize it for their own needs.
>
> It's pretty ambitious work so while I've pencilled this in for this month it's likely to roll over into subsequent months, and be likely something that would continue to evolve even after 1.0 is made.
>
> This work will directly impact vsgXchange, requiring internals of the OSG and Assimp loaders to be rewritten. This will also open the door to addressing some of the bugs/limitations in the present OSG loader, so rather than try to fix these OSG loader problems first, I'll do the general purpose work first, then port the OSG loader across and address the problems then.
>
> I will start up a separate thread for this work so I'd ask you to keep you comments/suggestions on this work to that thread.
>
> --
>
> When posting suggestions here about issues or new features you'd like tackled before VulkanSceneGraph-1.0 can you provide some input on the priority you see it. I find the broad priority "MUST HAVE," "Should have", "could have" demarcation useful when discussing priorities.
>
> One typically tackles the "MUST HAVE" items first, then you assess the timeline to see what "Should have" or even "could have" items could be tackled for the 1.0 releases.
>
> When making suggestions please also let us know what assistance, in manpower or funding, you can provide in developing and testing the features. This applies not only to your own preferred items but also ones that others a proposing. The more we can help each other the quicker we'll get across the 1.0 finishing line.
>
> Thanks for you input and help over the next few months,
> Robert.
>
>

[Tim Moore]

Hi,

I've been working with Pelican for several months on prototyping and evaluating a possible port of osgEarth to VSG, as well as other approaches. The experience has been mostly pleasant. Here are some things that I wish were different and that would be nice for VSG 1.0.

osgEarth  makes extensive use of pretty much every feature of OpenSceneGraph, from the math, geometry, and texture objects up through extensive customization of the scene graph and database paging. There's no way that I would jump in and change all of this at once, if ever. Instead, we keep as much OSG code around as possible and treat it as an "intermediate" format, converting it to the VSG scene graph as needed. I suspect that many sophisticated OSG users will use the same strategy. We do construct the basic terrain directly in VSG. We don't create any OpenGL contexts (that I know of :)) and might eventually get bitten by code that wants to know about capabilities, but so far this has worked out. With that in mind:

On Mon, Jan 3, 2022 at 12:19 PM Robert Osfield <robert....@gmail.com> wrote:

• Support for integration with OpenGL/OSG applications via EXT_external_object & VK_KHR_external_memory

Unless you have a specific usage in mind (or a paying client:)), I would make this low priority. It is going to be very squirrelly to get right, quantify performance, work around driver bugs, etc. I would much rather see effort put into refining vsgXchange for in-memory OSG scenegraph conversion, starting with an external interface to ConvertToVsg. 

In no particular order, here are changes that don't seem to impact the existing VSG ecosystem much:

Descriptor dstBinding and dstElement should not be properties of Descriptor. This complicates descriptor set creation and management. It seems much more intuitive to me that the binding come from the position of the Descriptor in the DescriptorSet::descriptors array.

It would be great if VSG implemented the reverse Z depth buffer scheme as described, among many other places, in https://developer.nvidia.com/content/depth-precision-visualized and https://vincent-p.github.io/notes/20201216234910-the_projection_matrix_in_vulkan/ . For the kinds of simulation and visualization apps targeted by VSG, this has the precision advantages of a logarithmic depth buffer without the hassle. On the other hand, in current VSG it is quite hard to properly implement this; it's impractical to customize the RecordTraversal to change the projection matrix. The cleanest way to do implement reverse Z is to change the VSG notion of clip coordinates to range from 1 to 0 near to far, and also change the default depth test. If you are receptive to this, I will implement it and submit a pull request.

It is unfortunate that RecordTraversal is not a Visitor and that it is currently tricky to create and insert a custom render traversal. Also unfortunate is the fact that the traversal object and its state are not passed to the record() functions of commands. Perhaps state isn't supposed to matter at that point...

The new StateSwitch node is great. Why not make the masks for it and Switch be uint64_t? We will inevitably want that, and due to alignment requirements it won't use any more memory in the nodes.

Tim

[Robert Osfield]

Hi Tim,

On Wed, 5 Jan 2022 at 12:45, Tim Moore <timo...@gmail.com> wrote:

I've been working with Pelican for several months on prototyping and evaluating a possible port of osgEarth to VSG, as well as other approaches. The experience has been mostly pleasant. Here are some things that I wish were different and that would be nice for VSG 1.0.

Cool. Any screenshots to share?

 

osgEarth  makes extensive use of pretty much every feature of OpenSceneGraph, from the math, geometry, and texture objects up through extensive customization of the scene graph and database paging. There's no way that I would jump in and change all of this at once, if ever. Instead, we keep as much OSG code around as possible and treat it as an "intermediate" format, converting it to the VSG scene graph as needed. I suspect that many sophisticated OSG users will use the same strategy. We do construct the basic terrain directly in VSG. We don't create any OpenGL contexts (that I know of :)) and might eventually get bitten by code that wants to know about capabilities, but so far this has worked out. With that in mind:

While I can see the appeal of hybrid approach, I suspect performance will be little better than just using OpenSceneGraph/OpenGL. 

On Mon, Jan 3, 2022 at 12:19 PM Robert Osfield <robert....@gmail.com> wrote:

• Support for integration with OpenGL/OSG applications via EXT_external_object & VK_KHR_external_memory

Unless you have a specific usage in mind (or a paying client:)), I would make this low priority. It is going to be very squirrelly to get right, quantify performance, work around driver bugs, etc. I would much rather see effort put into refining vsgXchange for in-memory OSG scenegraph conversion, starting with an external interface to ConvertToVsg. 

Tom Hogarth did an early implementation of the OSG->VSG via EXT_external_object & VK_KHR_external_memory, this was working under Windows for Tom, but I couldn't get it to work straight away under Linux - I had lots of other tasks to complete at the time so had to move on.  The VSG side of this work should be pretty minimal, most of the code will be in application/OSG/OpenGL glue side. 

It's interesting that you peg this as a lower priority.  Personally I had it in the "could have" for 1.0, but kept it into the ROADMAP.md as it had been there since the very early days of the VSG.

W/r/t providing a public vsgXchange::ConvertToVsg() funcation.  Originally the osg2vsg library had this, when I integrated this directly into osg2vsg I hide these functions.  I'm open to them being made public in a limited way - we have a have a graceful way to handle vsgXchange not being built against the OSG.

 

In no particular order, here are changes that don't seem to impact the existing VSG ecosystem much:

Descriptor dstBinding and dstElement should not be properties of Descriptor. This complicates descriptor set creation and management. It seems much more intuitive to me that the binding come from the position of the Descriptor in the DescriptorSet::descriptors array.

Interesting idea.  The choice of dstBinding and dstElement being properties of Descriptor was made to localise the set up the VkWriteDescriptorSet.  To follow your suggestion the dstBinding and dstElement values of VkWriteDescriptorSet would be done by DescriptorSet, while the pImageInfo/pBufferInfo/pTexelBufferView would be done by the Descriptors as they are done right now.

 

This change would require the public interface of the DescriptorSet and Descriptor classes to be changed as well as the implementation, this would in turn require mods to all vsgXchange and vsgExample codes.  I guess this might take a day or two's work if it went smoothly.  If we are to try this out I would suggest it should be something done earlier rather than later, certainly well before 1.0 as the potential knock on for client code is quite high.

It would be great if VSG implemented the reverse Z depth buffer scheme as described, among many other places, in https://developer.nvidia.com/content/depth-precision-visualized and https://vincent-p.github.io/notes/20201216234910-the_projection_matrix_in_vulkan/ . For the kinds of simulation and visualization apps targeted by VSG, this has the precision advantages of a logarithmic depth buffer without the hassle. On the other hand, in current VSG it is quite hard to properly implement this; it's impractical to customize the RecordTraversal to change the projection matrix. The cleanest way to do implement reverse Z is to change the VSG notion of clip coordinates to range from 1 to 0 near to far, and also change the default depth test. If you are receptive to this, I will implement it and submit a pull request.

Another interesting suggestion.  I haven't yet experimented with it myself.  I only have so many hours in the day so if you could try out an implementation that would be helpful.

 

It is unfortunate that RecordTraversal is not a Visitor and that it is currently tricky to create and insert a custom render traversal. Also unfortunate is the fact that the traversal object and its state are not passed to the record() functions of commands. Perhaps state isn't supposed to matter at that point...

I have considered making RecordTraversal a subclass from ConstVisitor, and chose to have it as it's own dedicated traversal class for efficiency purposes - it avoids a virtual function call when handling the apply,  The RecordTraversal is the most commonly called traversal type so I have been keen to avoid overheads during this traversal.  Building a complex and big enough scene graph to really stress this aspect of performance will be required to tease out whether adding an extra virtual function call will be an issue.

The Command::record() was originally conceived to be a very low level wrapper for code that just does a straight Vulkan call, so just takes the CommandBuffer that the Vulkan commands should be recorded to.  I guess one could change to having Command::record() take a RecordTraversal& rather than a CommandBuffer& and get the command buffer from the Record Traversal.  Again we need to be wary of performance implications of adding lockups.

If do make changes like this then we need to create some large test datasets to use a stress test to make sure we aren't hobbling performance in the process in making classes more extensible.

The new StateSwitch node is great. Why not make the masks for it and Switch be uint64_t? We will inevitably want that, and due to alignment requirements it won't use any more memory in the nodes.

When I implemented the vsg::StateSwitch state command and the vsg::Switch node I did reflect on whether uint64_t would be more appropriate than uin32_t.  My plan has been to raise this for discussion but you've now beat me to it :-)

So... community what do you think?

Cheers,

Robert.

 

[Tim Moore]

On Wed, Jan 5, 2022 at 4:29 PM Robert Osfield <robert....@gmail.com> wrote:

Hi Tim,

On Wed, 5 Jan 2022 at 12:45, Tim Moore <timo...@gmail.com> wrote:

It would be great if VSG implemented the reverse Z depth buffer scheme as described, among many other places, in https://developer.nvidia.com/content/depth-precision-visualized and https://vincent-p.github.io/notes/20201216234910-the_projection_matrix_in_vulkan/ . For the kinds of simulation and visualization apps targeted by VSG, this has the precision advantages of a logarithmic depth buffer without the hassle. On the other hand, in current VSG it is quite hard to properly implement this; it's impractical to customize the RecordTraversal to change the projection matrix. The cleanest way to do implement reverse Z is to change the VSG notion of clip coordinates to range from 1 to 0 near to far, and also change the default depth test. If you are receptive to this, I will implement it and submit a pull request.

Another interesting suggestion.  I haven't yet experimented with it myself.  I only have so many hours in the day so if you could try out an implementation that would be helpful.

I've created a pull request that implements this. It's a small change. It impacts a couple of examples  (vsgheadless, vsgrendertotexture) that explicitly specify a depth clear value. If you accept this, the convention should be well-documented somewhere. What are your thoughts for VSG 1.0 documentation?

Tim

[Robert Osfield]

Hi Tim,

On Fri, 7 Jan 2022 at 17:26, Tim Moore <timo...@gmail.com> wrote:

It would be great if VSG implemented the reverse Z depth buffer scheme as described, among many other places, in https://developer.nvidia.com/content/depth-precision-visualized and https://vincent-p.github.io/notes/20201216234910-the_projection_matrix_in_vulkan/ . For the kinds of simulation and visualization apps targeted by VSG, this has the precision advantages of a logarithmic depth buffer without the hassle. On the other hand, in current VSG it is quite hard to properly implement this; it's impractical to customize the RecordTraversal to change the projection matrix. The cleanest way to do implement reverse Z is to change the VSG notion of clip coordinates to range from 1 to 0 near to far, and also change the default depth test. If you are receptive to this, I will implement it and submit a pull request.

Another interesting suggestion.  I haven't yet experimented with it myself.  I only have so many hours in the day so if you could try out an implementation that would be helpful.

I've created a pull request that implements this. It's a small change. It impacts a couple of examples  (vsgheadless, vsgrendertotexture) that explicitly specify a depth clear value. If you accept this, the convention should be well-documented somewhere.

Thanks I'm in the middle of work Camera usage, once that's wrapped up I'll do a review. 

What are your thoughts for VSG 1.0 documentation?

Nothing is set in stone.  I'd like to make sure most classes have a description, unless it's super obvious, and where needed methods/members get a description.  Once we've fleshed out the examples for 1.0 going back over the VSG itself to provide the appropriate links from core VSG class docs to relevant vsgExamples should help users.

I have experimented with README.md's in the VSG code base to help browsing the codebase, but I'm not sure how effective this is.

My priority at this point is shaking down the API and implementation to get them to a stable enough of place for widespread testing, usage and at this time documentation will become more important.  I'm not a great educator so would welcome help in this effort.

Cheers,

Robert.

[Robert Osfield]

I have merged Tim's work on reverse depth as a branch of VulkanSceneGraph:

    https://github.com/vsg-dev/VulkanSceneGraph/tree/timoore-reverse-depth

We'll need to update any vsgExamples that are affected by these changes.  Once we've done these updates and general testing we can look at merged with VSG master.  Feedback on this very welcome.