VulkanSceneGraph almost ready for 1.0, please test/comment

[Robert Osfield]

Hi All,

Over the last two weeks I've finished up all the significant work that had been ongoing for the last couple of months, this is all now merged with VSG master so I've just updated the version to 0.6.0 to mark this point in time.  I consider this a pre-release for 1.0.

I am aware of a problems with LODs and normal computation in the assimp related shaders when handling scaled models so I'll be looking into these two issues.  I don't consider them show stoppers.

There are issues in osg2vsg's mapping of OSG scene graphs to VSG scene graphs which I'll like to resolve, but as this is an ancillary project I don't consider them as show stopper for the VSG-1.0 release as all the required fixes will be within osg2vsg.

My plan is do a code review over the next week to spot areas where API is inconsistent or needs tightening up. I would appreciate the community doing the same and reporting areas that look like they need further review here on this thread.  Doesn't matter how trivial you think it is just let us know.

As well as review of the VSG code base, testing on as many build and hardware platforms will be really important, as is testing against your own applications.  If there are oddities and bugs that you come across let us know on this thread, or via the Issue tracker.

If by the end of next week the code base is looking solid then I'll go for a final 1.0 release in final week of October (24th onwards.)

Thanks in advance with testing and shaking down the VSG,

Robert.

[kan...@nodein.com]

Hi Robert,

I suggest a vcpkg port file... I am happy to contribute and create a port file. I just got started with vsg, but do not wish to hold up 1.0.

Suresh.

[ca...@imap.cc]

Hi Robert,

My new iMac showed up today and I decided to test all of the examples. The majority of them ran fine but there are few reporting errors. I listed the problematic examples below with the demo run and the output. I'm not sure how helpful the below information is but I thought I would post. Perhaps I ran some of them wrong?

Vsgcompute:

calebcohoon@calebs-iMac ~ % ~/Workspace/vsgExamples/bin/Debug/vsgcompute

[mvk-error] VK_ERROR_INITIALIZATION_FAILED: Shader library compile failed (Error code 3):

program_source:26:26: error: 'function_constant' has a duplicate index '2'

constant uint _167_tmp [[function_constant(2)]];

                         ^

program_source:24:26: note: duplicate 'function_constant' index '2' here

constant uint _166_tmp [[function_constant(2)]];

                         ^

.

[mvk-error] VK_ERROR_INVALID_SHADER_NV: Compute shader function could not be compiled into pipeline. See previous logged error.

libc++abi: terminating with uncaught exception of type vsg::Exception

zsh: abort      ~/Workspace/vsgExamples/bin/Debug/vsgcompute

Vsgcomputevertex:

calebcohoon@calebs-iMac ~ % ~/Workspace/vsgExamples/bin/Debug/vsgcomputevertex 

[mvk-error] VK_ERROR_OUT_OF_DEVICE_MEMORY: Could not allocate a host-coherent or exportable VkDeviceMemory of size 0 bytes. The maximum memory-aligned size of a host-coherent VkDeviceMemory is 8589934592 bytes.

libc++abi: terminating with uncaught exception of type vsg::Exception

zsh: abort      ~/Workspace/vsgExamples/bin/Debug/vsgcomputevertex

Vsgdeviceselection:

calebcohoon@calebs-iMac ~ % ~/Workspace/vsgExamples/bin/Debug/vsgdeviceselection

vkEnumerateInstanceVersion() 4206823

VK_API_VERSION = 1.3.231.0

Using vsg::Window default behavior to create the required vsg::Device.

Warning: ObjectFactory::create() failed to find means to create object

Warning: Unable to create instance of class : ???'2dA?? 5???/earth_root_L0_X0_Y0/ear

Warning: ObjectFactory::create(_Y0_subtile.vsgb?t?\??`?) failed to find means to create object

Warning: Unable to create instance of class : _Y0_subtile.vsgb?t?\??`?vsg::MatrixTransform

vsg::Groupvsg::StateGrouvsg::VertexIndexDraw    vsg::vec3Array

                                                              \2???U??/??2??ʡ?f??2??ʟ?m!???2????m?/!???2?????/??!?

                                  ??Һ/?????x

????/????2?????/!???2???M?5/???-?9(0hq??\+??W#0??j?2???:/!???2???M?5??ʐ8@˕[Q0??ɱ?=?vrK0/:??2???:?!???2????̯?ʐ8@˕[Q0??I??=?vrK0/:?I2????ү!???2???U??????-?9(0hqJ?\+??W#0??jJ?}˒c?J??I?}˒c?J???f?˧DnJhqJf?˧DnJhq?]i?ʺ

vsgdynamictexture_cs:

calebcohoon@calebs-iMac ~ % ~/Workspace/vsgExamples/bin/Debug/vsgdynamictexture_cs models/lz.vsgt

[mvk-error] VK_ERROR_INITIALIZATION_FAILED: Shader library compile failed (Error code 3):

program_source:21:25: error: 'function_constant' has a duplicate index '2'

constant uint _81_tmp [[function_constant(2)]];

                        ^

program_source:19:25: note: duplicate 'function_constant' index '2' here

constant uint _80_tmp [[function_constant(2)]];

                        ^

.

[mvk-error] VK_ERROR_INVALID_SHADER_NV: Compute shader function could not be compiled into pipeline. See previous logged error.

libc++abi: terminating with uncaught exception of type vsg::Exception

zsh: abort      ~/Workspace/vsgExamples/bin/Debug/vsgdynamictexture_cs models/lz.vsgt

Vsgmeshshader: I couldn't run due to no support from GPU

Vsgfraytracing: I couldn't run due to no support from GPU

Vsgtexturearray:

calebcohoon@calebs-iMac ~ % ~/Workspace/vsgExamples/bin/Debug/vsgtexturearray models/skybox.vsgt 

[mvk-error] VK_ERROR_DEVICE_LOST: MTLCommandBuffer "vkQueueSubmit CommandBuffer on Queue 0-0" execution failed (code 1): Discarded (victim of GPU error/recovery) (00000005:kIOGPUCommandBufferCallbackErrorInnocentVictim)

libc++abi: terminating with uncaught exception of type vsg::Exception

zsh: abort      ~/Workspace/vsgExamples/bin/Debug/vsgtexturearray models/skybox.vsgt

[Rainer Gericke]

Hi All,

I have made a complete test run of the vsg examples on macOS 13 and Windows 11. Both machines with physical GPU and emulated GPU (Swiftshader). You find the results in the attached Excel table.

MacOS Vulkan generally does not support compute shaders and meshshaders. MacOS Metal 3d api has raytracing implemented, but it is actually impossible to use it with Vulkan. Vsgtexturearray should work yet, but it doesn't due to a driver bug.

Generally there is actually a problem with files load via curl (vsghelloworld). The file seems to be loaded, but the content strands as binary data on the terminal, no graphical output occurs.

[Robert Osfield]

Thanks the extensive testing, kudos to Rainer for the spreadsheet this is really useful.

In my own testing I have found a regression in vsgcomputevertex and vsgdynamictexture_cs with them both crashing with what looks to be an allocation of zero sized buffer.  I don't know the cause of this regression so will now look into this.

[ca...@imap.cc]

Yeah kudos on the great testing spreadsheet!

[Robert Osfield]

I have used git bisect to track down the commit that caused the regression in vsgcomputevertex - all the way back on September 13th:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/779d47be9a93759a64b32ca40ad118a6935a0bf1

I may be able to resolve this issue today but it's already late afternoon, almost evening so I'm going to not attempt to bust a gut. Spotting a regression this near to tagging a release is a sign - the code base isn't ready for release.  Even if I got a fix in now it'd need to be tested by the community, as with all the other examples/3rd party libs and applications.

So officially I'm delaying the VSG-1.0 into next week.  Once I've got this fix checked in and merged my other cleanup of TODO items merged with VSG master I'll reasses how things are converging to VSG-1.0 and set a new target date for the release.

Arrgggghhhhh.... Writing software is hard :-)

Thanks to everyone for your testing and patience.

[Robert Osfield]

On Sat, 5 Nov 2022 at 17:31, Robert Osfield <robert....@gmail.com> wrote:

I have used git bisect to track down the commit that caused the regression in vsgcomputevertex - all the way back on September 13th:

    https://github.com/vsg-dev/VulkanSceneGraph/commit/779d47be9a93759a64b32ca40ad118a6935a0bf1

I have resolved the cause of the crash in vsgcomputevertex and have a quick fix to gets things working but need to do a wide review of the TransferTask.cpp to see what is the best fix. 

I should get the final fix check later today.  Right now I have a dog to walk!

[Robert Osfield]

On Sunday, 6 November 2022 at 13:54:44 UTC Robert Osfield wrote:

I should get the final fix check later today.  Right now I have a dog to walk!

Back from walk and checked in a very simple fix:

    https://github.com/vsg-dev/VulkanSceneGraph/pull/558

[Robert Osfield]

My plan is to create an OpenSceneGra[h-1.0 branch tomorrow.  We can then use that branch for further testing and to make the final release from, and any subsequent patch releases.

["François Cami"]

On Sun, Nov 6, 2022 at 7:36 PM Robert Osfield <robert....@gmail.com> wrote:
>
> My plan is to create an OpenSceneGra[h-1.0 branch tomorrow. We can then use that branch for further testing and to make the final release from, and any subsequent patch releases.

Could you create a 1.0RC1 (or similar) tag, possibly from the first
commit of that branch? This would help for downstream purposes.

> --
> You received this message because you are subscribed to the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vsg-users+...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAFN7Y%2BUd_GLTewO04_L2%2B5mR8_tGGAvp9Qjcd_5cT_uks0PACw%40mail.gmail.com.

[Robert Osfield]

On Sun, 6 Nov 2022 at 19:16, "François Cami" <f...@fcami.net> wrote:

On Sun, Nov 6, 2022 at 7:36 PM Robert Osfield <robert....@gmail.com> wrote:
>
> My plan is to create an OpenSceneGra[h-1.0 branch tomorrow.  We can then use that branch for further testing and to make the final release from, and any subsequent patch releases.

Could you create a 1.0RC1 (or similar) tag, possibly from the first
commit of that branch? This would help for downstream purposes.

That's my plan - it's how I've done it with the OSG over the years.  The CMake support is already set up and awaits me testing it for real :-)

 

["François Cami"]

Awesome!
Could you please tag https://github.com/vsg-dev/osg2vsg as well?
Either in-sync with the 1.0.0-rcX VSG tag or with a lower number if
deemed not ready.

BTW, tag-generated tarballs result in weird directories:
https://github.com/vsg-dev/vsgXchange/archive/refs/tags/vsgXchange-1.0.0-rc2.tar.gz
contains vsgXchange-vsgXchange-1.0.0-rc2/ (the name is present twice).
I don't know what's causing this.

[Robert Osfield]

On Fri, 11 Nov 2022 at 13:30, "François Cami" <f...@fcami.net> wrote:

On Sun, Nov 6, 2022 at 8:51 PM Robert Osfield <robert....@gmail.com> wrote:
Could you please tag https://github.com/vsg-dev/osg2vsg as well?

Either in-sync with the 1.0.0-rcX VSG tag or with a lower number if
deemed not ready.

osg2vsg isn't ready for it's own 1.0. I'd like to rewrite the OSG->VSG code as the VSG now have features that were present in the early days of the VSG project so it will be possible to streamline the code and make it more feature complete.

Perhaps 0.1.0 tag?

Same applies vsgimGui and vsgQt.  My main focus is VSG, vsgXchange, vsgExamples for now.

 

BTW, tag-generated tarballs result in weird directories:
https://github.com/vsg-dev/vsgXchange/archive/refs/tags/vsgXchange-1.0.0-rc2.tar.gz
contains vsgXchange-vsgXchange-1.0.0-rc2/ (the name is present twice).
I don't know what's causing this.

github is creating these tarballs, no idea why it might choose to name it twice.

 

Perhaps there is a setting?  Happy to take input.

 

[Robert Osfield]

Hi All,

Today I spent the day looking at out of source build and vsgFramework build combinations, looking to address an issue with uninstall with out of source builds and an issue with assimp not being picked up in a vsgFramework.  It feels like I've done about a 1000 test builds today, I'm just going about crazy,

The out of source uninstall issue was fixed as part of PR that Ralf Habacker provided yesterday but I was hesitant to merge something that tweaked the CMake build side this close to a release.  In the end I decided to the merge the PR into VSG master, and merged the companion VSG projects that Ralf also provided PR's for as part of these CMake/vsgMacro changes.  The change to the VSG were:

    https://github.com/vsg-dev/VulkanSceneGraph/pull/566

Changes of vsgXchange, vsgImGui, vsgQt, vsgExamples were:

    https://github.com/vsg-dev/vsgXchange/pull/129

    https://github.com/vsg-dev/vsgImGui/pull/28

    https://github.com/vsg-dev/vsgQt/pull/16

    https://github.com/vsg-dev/vsgExamples/pull/185

To address the problem with vsgFramework building Assimp but vsgXchange not being able to find it due it's FetchContent support not aware of this type of build usage, I had to put in local set up of the assimp::assimp lib and associated CMake vars, and added in dependencies to ensure builds happening in the required order:

    https://github.com/vsg-dev/vsgXchange/pull/129

The automated builds are working fine across platforms, and the testing things look OK to me, but my brain has started turning inside out with build testing a number of separate projects and dependency combinations... so I probably can't be trusted to tie my laces at this point...

So please, could everyone test the VSG, vsgXchange and vsgExamples master, and any other associated projects you can (including your own :-)  

If things look solid with the respective masters then I'll merge these into the respective 1.0 branches and make another release candidate.  If I can give positive feedback today I'll do the merge with 1.0 branches and tag the next rc this evening.  Failing that I'll do that tomorrow morning.

--

My plan is to make the release on Sunday early evening during the Zoom 1.0 Release party.  I'm thinking of holding the Zoom session from 20:00 GMT for an hour.  It'll be an open invite.  If you can present some of your own work then this would be awesome.

Let me know if an earlier/later session would work better for you.  I'm trying to find a time that has a reasonable overlap with other timezones.

Cheers,

Robert.

["François Cami"]

On Fri, Nov 11, 2022 at 3:39 PM Robert Osfield <robert....@gmail.com> wrote:
>
> On Fri, 11 Nov 2022 at 13:30, "François Cami" <f...@fcami.net> wrote:
>>
>> On Sun, Nov 6, 2022 at 8:51 PM Robert Osfield <robert....@gmail.com> wrote:
>> Could you please tag https://github.com/vsg-dev/osg2vsg as well?
>>
>> Either in-sync with the 1.0.0-rcX VSG tag or with a lower number if
>> deemed not ready.
>
>
> osg2vsg isn't ready for it's own 1.0. I'd like to rewrite the OSG->VSG code as the VSG now have features that were present in the early days of the VSG project so it will be possible to streamline the code and make it more feature complete.
>
> Perhaps 0.1.0 tag?

0.1.0 works for me, though if you'd rather see it not distributed
downstream yet, maybe no tag works better.

> Same applies vsgimGui and vsgQt. My main focus is VSG, vsgXchange, vsgExamples for now.

Understood.

>> BTW, tag-generated tarballs result in weird directories:
>> https://github.com/vsg-dev/vsgXchange/archive/refs/tags/vsgXchange-1.0.0-rc2.tar.gz
>> contains vsgXchange-vsgXchange-1.0.0-rc2/ (the name is present twice).
>> I don't know what's causing this.
>
>
> github is creating these tarballs, no idea why it might choose to name it twice.
>
> Perhaps there is a setting? Happy to take input.
>
>

> --
> You received this message because you are subscribed to the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vsg-users+...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/CAFN7Y%2BXTKmkN3E9qHA0mNMe5-%2BOe%3D80xnwRCcYd6_OKperP3Eg%40mail.gmail.com.

[Robert Osfield]

On Fri, 11 Nov 2022 at 17:39, "François Cami" <f...@fcami.net> wrote:

0.1.0 works for me, though if you'd rather see it not distributed
downstream yet, maybe no tag works better.

Yea, I wouldn't recommend osg2vsg being distributed in binary form with 3rd party software repositories as it still needs more work to be production ready.

 

VSG, vsgXchange and vsgExamples are in good shape for 1.0 so I am happy for it to be more widely distributed. Personally I'd rather have the VSG and associated components so easy to compile that engineers will compile from source and link it directly in their apps.  Static linking helps this.

Cheers,

Robert.

[Robert Osfield]

I have tagged VulkanSceneGraph-1..0.0-rc4, vsgXchange-1.0.0-rc3 and vsgExamples-1.0.0-rc3:

    https://github.com/vsg-dev/VulkanSceneGraph/tree/VulkanSceneGraph-1.0.0-rc4

    https://github.com/vsg-dev/vsgXchange/tree/vsgXchange-1.0.0-rc3

    https://github.com/vsg-dev/vsgExamples/tree/vsgExamples-1.0.0-rc3

Please test :-)

[Robert Osfield]

Hi All,

Today I made some revisions to the VSG to tighten up the licensing notice, and to remove settings code originally from GLSLang replacing it with something I wrote.  I've merged this changes with VSG master  and the VulkanSceneGraph-1.0 branch, and have made a new rc:

    https://github.com/vsg-dev/VulkanSceneGraph/tree/VulkanSceneGraph-1.0.0-rc5

The only change to vsgXchange today was the addition of find_dependency(Freetype) to the vsgXchangeConfig.cmake, this is merged with master and the 1.0 branch, and the new rc for it:

   https://github.com/vsg-dev/vsgXchange/tree/vsgXchange-1.0.0-rc4

vsgExamples also just got one change - the addition of make install support for the data directory which now gets placed in CMAKE_INSTALL_PREFIX/share/vsgExamples/data.  This is merged with master and the 1.0 branch, and the rc is:

   https://github.com/vsg-dev/vsgExamples/tree/vsgExamples-1.0.0-rc4

It's late here now so I'll head off line.  If everything looks OK tomorrow morning I'll set up the Zoom call for 8pm GMT, with the plan to tag the release during the call just to add a bit of jeopardy!

Cheers,

Robert.

[Rainer Gericke]

Hi All,

here is my last result matrix for Mac and Windows 11.

[Robert Osfield]

Thanks for the testing and spreadsheet Rainer.  Do you use a script to run and collate all these tests, or are you methodically going through by hand?

Could you provide some more details on the strange results you see? Thanks.

--

As an attempt at resolving the libpng dependency found on some freetype installs I've changed vsgXchange's freetype support to check for a modern freetype-config.cmake file and if that fails fallback to the old style FindFreetype.cmake usage.  This is checked into the vsgXchange's freetype_config branch:

    https://github.com/vsg-dev/vsgXchange/tree/freetype_config

If this works across platforms then I'll merge the vsgXchange master and the vsgXchange-1.0 branch.

Cheers,

Robert.

[Rainer Gericke]

I called all examples by hand. Here are the most important results:

vsgexecutecommands:

 
vsgsubpass:

there weirdest output comes from vsgdeviceselection: no window but a big bunch of binary data on the terminal (only the first ten 'lines'):

[Robert Osfield]

Hi Rainer,

On Sun, 13 Nov 2022 at 11:36, Rainer Gericke <dr.ratz...@gmail.com> wrote:

I called all examples by hand. Here are the most important results:

vsgexecutecommands:

This looks like the viewport and the window size out out of sync in someway.  Does this make have some of resolution scaling enabled?

Does this happen for other VSG apps?

 

vsgsubpass:

The actual rendered part looks normal, but like vsgexecutecommands it looks like there is mismatch between rendered viewport and the whole window size is problematic.

 

there weirdest output comes from vsgdeviceselection: no window but a big bunch of binary data on the terminal (only the first ten 'lines'):

I have just tried vsgdeviceselection and found the same.  I don't normally run it without command line options.

What it looks like to me is the the database it attempts to load:

    https://raw.githubusercontent.com/robertosfield/TestData/master/Earth_VSG/earth.vsgb

It's a test dataset I created a long while back, and I presume it contains outdated binary format which is no longer supported. 

I have changed the default file path to modles/lz.vsgt, and checked this into vsgExamples master and the vsgExamples-1.0 branch.

Cheers,

Robert.

 

[Rainer Gericke]

vsgdeviceselection shows models/lz.vsgt now after providing the commit, so there are four more 'yes' in the matrix, thanks.

The recent Macs all have 'retina displays', this means if you open a window with 800x600 pixels, you get an internal canvas of 1600x1200 pixels. That's also the reason, why the screenshots on the Mac are so big. The native Mac window in VSG activates this behavior as default and VSG normally handles it correctly with pretty good grapic quality. On Windows 11, this is not an issue, but the windows of vsgexecutecommands are also very small, like the rendered parts on MacOS, and one of it is under the Menu line, and cannot be moved by the user. 

The native Windows VSG window doesn't rescale the viewport after a window rescale event, seen at the vsgtext example, shrinking the window cuts off a part of the 3d text, instead of shrinking its size too. 

[Rainer Gericke]

The issue with vsgdeviceselection brought me to the idea to look into the source code of vsghelloworld, it showed the same strange behavior, because it load the same incompatible file. This could be healed by providing a valid file on the webserver.

[Robert Osfield]

Hi Rainer,

Are you running the Windows tests on the Mac?

Robert 

--

You received this message because you are subscribed to the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vsg-users+...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/da2f3c5c-343f-4247-9fc3-920dfb660ee1n%40googlegroups.com.

[Rainer Gericke]

No, thats not possible any more, I have a Windows notebook for this.

[Rainer Gericke]

Here is the vsgexecutecommands example on windows, after start the menu line of the above window is outside of the screen and cannot be reached. After resizing the window, the menu line appears again, but the viewport is not scaled apparently.

[Rainer Gericke]

Here is the same with resized window.

[Robert Osfield]

Hi Rainer,

Thanks for all info.  I have been out all afternoon visiting one of my daughters at uni so wasn't able to follow up right away.  I am back and tidying up loose ends, hopefully :-)

I have fixed the vsghelloworld example so it now loads models/openstreetmap.vsgt model - a model not around when I originally wrote this example.  This fix is now merged with vsgExamples master and vsgExamples-1.0 branches.

The resize issue with the vsgexecturecommands example is something we will need to investigate after the release.

I'm now setting up a script to do the 3 releases one after the other so I don't mess up live!

Robert.