New vsg::PagedLOD/vsgXchange::curl/ReadyMap/OpenStreetMap example

[Robert Osfield]

I have just cheeked in a new vsgpaged example to vsgExamples:

   https://github.com/vsg-dev/vsgExamples/tree/master/examples/nodes/vsgpagedlod

This example is illustrates how to use vsg::PagedLOD and a custom vsg::ReaderWriter to stream whole earth data from either ReadyMap or OpenStreetMap servers.  It only took 674 lines of code to put together.

While only a modest amount of code in the example it's the culmination of all work on vsg::DatabasePager/vsg::PagedLOD,  vsgXchange image loaders and new vsgXchange::curl loader for downloading http and https data.

    https://www.youtube.com/watch?v=Pdbsb6mj8_Y

The ReadyMap is loaded by default and can seen simply by running:

    vsgpagedlod

To view the OpenStreetMap data add the --osm command line option:

   vsgpagedlod --osm

Home!!

I haven't loaded any height field data in these examples, thought this shouldn't require too much more work.

Another item for future development would be a new camera manipulator that is suitable for handling whole earth databases as the vsg::Trackball really struggles to work smoothly when close to the earths surface.

[Gavin McIntosh]

Getting more useful every day.

This will be handy, just ordered a GPS module this week.

I need to fix that Raspberry Pi "seg fault" error on the original Earth example, the new online version also has a seg fault error.

Should try this on my Window10 PC, it has real Vulkan supported video cards.

Regards

Gavin

[Robert Osfield]

On Wed, 17 Mar 2021 at 02:23, Gavin McIntosh <g.mci...@griffith.edu.au> wrote:

This will be handy, just ordered a GPS module this week.

I need to fix that Raspberry Pi "seg fault" error on the original Earth example, the new online version also has a seg fault error.

Does it seg fault straight away?  Could you provide a stack trace.

 

Should try this on my Window10 PC, it has real Vulkan supported video cards.

Do you have the VSG running on your Windows PC?

[Robert Osfield]

Out of curiosity I set up a camera animation path and osgEarth.earth fle to render the OpenStreetMap data and run osgvieer and vsgpagelod to see how they both performed.  The osgEarth rendering approach differs from the vsgpagedlod one so it's not an apples to apples comparison.  The osgEarth renders the tiles blending between levels but selects lower levels of detail compared to the vsgpagedlod example.  The file caches are used in both instances, with the VSG file cache ending up with around 5 x more tiles than the osgEarth one underling the difference in LOD selection.  I cant' say which is a bigger cost - blending or rendering at a higher LOD.

The osgEarth run averages 878fps while the vsgpagedlod one averages 2598 fps on my Geforce 2060/AMD Ryzen 7 2700 system.  The OSG is running multi-threaded while the VSG was running single threaded.

While it's not a perfect like for like comparison the VSG is still nearly 3 x faster, extra blending isn't going to explain that level of difference away.

[Robert Osfield]

Looks like I missed putting the link to the youtube video I recorded with the osgEarth vs vsgpagdlod comparison, so here it is:

    https://www.youtube.com/watch?v=nOQxr09ald4

[Gavin McIntosh]

Hi Robert

Nice comparison.

vsgEarth.org soon?

The performance difference on Vulkan is showing results.

If a change from 20fps to 60fps happens on Pi's it makes something painful useful.

Are there online 3D maps suitable for testing vsg?

I had forgotten about osgEarth.

I think I tried it years ago and got nowhere on the Pi, but that was back in Pi3B+ days.

Not sure if my 2019 FlightGear testing included osgEarth?

Hmm, it was the old version 18.3.2.

These will test the Pi's and my compiling/debugging skills.

Some rumours Seg fault is the Pi's uSD failing, so will try on multiple cards/USB sticks n SSDs to rule that issue out.

Time for a rotating metal file system?

Testing vsg on a 32bit Pi OS last night, vsg and vsgExamples compiled, vsgXchange gave a vsgconv - osg node error.

Will need to check osg, not sure if it is happy on 32bit.

Regards

Gavin

---

From: vsg-...@googlegroups.com <vsg-...@googlegroups.com> on behalf of Robert Osfield <robert....@gmail.com>
Sent: Thursday, March 18, 2021 5:13 AM
To: vsg-users : VulkanSceneGraph Developer Discussion Group <vsg-...@googlegroups.com>
Subject: Re: [vsg-users] Re: New vsg::PagedLOD/vsgXchange::curl/ReadyMap/OpenStreetMap example

 

Out of curiosity I set up a camera animation path and osgEarth.earth fle to render the OpenStreetMap data and run osgvieer and vsgpagelod to see how they both performed.  The osgEarth rendering approach differs from the vsgpagedlod one so it's not an apples to apples comparison.  The osgEarth renders the tiles blending between levels but selects lower levels of detail compared to the vsgpagedlod example.  The file caches are used in both instances, with the VSG file cache ending up with around 5 x more tiles than the osgEarth one underling the difference in LOD selection.  I cant' say which is a bigger cost - blending or rendering at a higher LOD.

The osgEarth run averages 878fps while the vsgpagedlod one averages 2598 fps on my Geforce 2060/AMD Ryzen 7 2700 system.  The OSG is running multi-threaded while the VSG was running single threaded.

While it's not a perfect like for like comparison the VSG is still nearly 3 x faster, extra blending isn't going to explain that level of difference away.

--
You received this message because you are subscribed to a topic in the Google Groups "vsg-users : VulkanSceneGraph Developer Discussion Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vsg-users/SyqjmJQ_Io4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vsg-users+...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vsg-users/e4870839-5f70-418d-9f6f-110b23925180n%40googlegroups.com.

[Robert Osfield]

Hi Gavin,

On Thu, 18 Mar 2021 at 03:13, Gavin McIntosh <g.mci...@griffith.edu.au> wrote:

Nice comparison.

vsgEarth.org soon?

The performance difference on Vulkan is showing results.

If a change from 20fps to 60fps happens on Pi's it makes something painful useful.

The performance delta will depend upon the bottleneck.  Both the OSG and VSG would get 60fps if I had vysnc on for the tests.

As to what performance delta you might expect on the Pi4 when running this type of tests, it may well be more than 3x faster than I saw on my desktop with fast CPU and GPU.  The CPU is major bottleneck for the OSG/OpenGL, so if you are struggling for CPU speed then the VSG/Vulkan will held back less and be able to achieve closer to the capability of the GPU rather than the CPU.

I've attached the .earth file if you want to do a comparison yourself - but would require osgEarth though...

 

Are there online 3D maps suitable for testing vsg?

That's what the vsgpagedlod example is all about - it by default uses the ReadyMap server http://readymap.com/ when you run:

  vsgpagedlod

And OpenStreetMap https://www.openstreetmap.org when you run:

   vsgpagedlod --osm

 

I had forgotten about osgEarth.

I think I tried it years ago and got nowhere on the Pi, but that was back in Pi3B+ days.

Not sure if my 2019 FlightGear testing included osgEarth?

Hmm, it was the old version 18.3.2.

I don't know the status of FlightGear w.r.t osgEarth.

 

These will test the Pi's and my compiling/debugging skills.

One thing that might be an issue is how many descriptor/descriptor sets that the vsgpagedlod is preallocating for the paged databases.  In vsgExamples/examples/nodes/vsgpaged/TileReader.cpp you'll find the lines:

    uint32_t maxLevel = 20;
    uint32_t estimatedNumOfTilesBelow = 0;
    uint32_t maxNumTilesBelow = 40000;

Changing maxNumTilesBelow from 40000 to something like 1000 might lower the up front allocations, there is a chance that the default I've chosen is simply too large.

Changing to a lower value will results in the VSG running out of descriptors sooner though, so it'd only be a temporary reprieve.  Longer term I will implement a dynamical scheme for allocating DescriptorPool on demand to handle paging in new tiles.

 

Some rumours Seg fault is the Pi's uSD failing, so will try on multiple cards/USB sticks n SSDs to rule that issue out.

Time for a rotating metal file system?

Solid state memory is lower power consumption and often faster so would will in terms of fit with a small form factor device, reliability does still seem to be with the spinning disks though...

 

Testing vsg on a 32bit Pi OS last night, vsg and vsgExamples compiled, vsgXchange gave a vsgconv - osg node error.

Will need to check osg, not sure if it is happy on 32bit.

Osg node error?  Could you elaborate.

 

It may be best to disable the compile of the OSG component of vsgXchange by setting the vsgXchange_OSG var to OFF using ccmake or to pass the value in to cmake on the command line i.e.

     cmake -DvsgXchange_OSG=OFF

vsgXchange has this control for vsgXchange_CURL, vsgXchange_GDAL, vsgXchange_OSG, vsgXchange_assimp and vsgXchange_freetype so you can toggle each one of them off even if cmake finds the associated dependency.

[Robert Osfield]

I have checked in an improvement to the vsgpagedlod examples TileReader class that improve tile read performance when streaming over the internet by 3 x, and local file cache by 50%.  Just a couple of dozen lines of extra code were required to leverage the support that I previously wrote for the multifile version of vsg::read(..) that leverages the vsg::OperationThreads class to read multiple files in parallel.  The changes to the vsgpagedlod example are:

    https://github.com/vsg-dev/vsgExamples/commit/af56973fbedab36b6b1028d83b79e6120f36dfa6

I've recorded a youtube video to illustrate the diffrences using a side by side comparison with vsgpagedlod configured to use an vsg::OperationThreads with 8 threads and one running with the standard thrading provided by vsg::DatabasePager (4 reading threads and 1 compile thread).

    https://www.youtube.com side by side comparison

Changes are checked into vsgExamples master.