How to reduce the cache size limit of OSG ?

[Hui Li]

Hi, 

I am using OSG to show captured point cloud at run time. The dimension of point cloud is 1920x1200. 

I keep capturing the point cloud from a 3D camera and display it with a osgViewer, but memory consumption  keeps growing until ~5GB. 

I know that OSG caches something to improve performance. But in my case, such an simple app consume a space like 5GB is not making sense, so I wonder how I could change the cache size to lower the memory usage. 

Thanks!

[OpenSceneGraph Users]

Hi??

There is nothing we can advice as you say nothing how you are managing your scene graph.  All we know is you are doing something with a point dataset and memory keeps growing.

You need to provide some source of how to you are setting up your scene graphs and how you are updating it.  Once we know what you are doing we'll be able recommend  a better way to do it without memory growth.

Robert.

[Hui Li]

Hi Robert, 

The codes looks like this: 

```

root = GroupOfSwitch(); 

viewer->setSceneData(root);

while (!viewer->done()) {

    root->removeChild(0); // Remove the point cloud 

    pc = sensor.Capture();  // Get the point cloud

    geo = CreateGeometryNode(pc);

    root->addChild(geo); // Add the pointcloud 

    viewer->frame();

}

```

Thanks, 

Hui

[Robert Osfield]

Hi Hui,

On Tuesday, 26 May 2020 10:36:18 UTC+1, Hui Li wrote:

The codes looks like this: 

```

root = GroupOfSwitch(); 

viewer->setSceneData(root);

while (!viewer->done()) {

    root->removeChild(0); // Remove the point cloud 

    pc = sensor.Capture();  // Get the point cloud

    geo = CreateGeometryNode(pc);

    root->addChild(geo); // Add the pointcloud 

    viewer->frame();

}

In "principle" this shouldn't cause unbounded memory growth but as you don't provide any details about the CreateGeometryNode() there is no way for use know what is going on there.  It could be that your leak it right there in that function call, or else where somewhere in your application.

While I can't comment about the details of your implementation the approach of removing, creating and adding new nodes and associated data each frame is really, really inefficient.

As much as possible you should try to reuse data.  For a point cloud application where the data is dynamic I would personally allocate a fixed number of Geometry nodes with fixed Vec3Array's within them and single DrawArray primitive set.  I'd allocate the Vec3Array to it's full size right up front and next change it's size.  The DrawArrays has a count value that you can set to specify how many vertices in the array need to be rendered and needn't be the whole length of the Vec3Aray.

With really large datasets I would chunk the data so rather one big osg::Geometry, I'd have say 10,000 vertices in each osg::Geometry.  I would also use VertexBufferObjects and avoid using DisplayLists.

Do this all correctly and you should get a good performance and no memory growth.

Robert.

 

[Hui Li]

Hi Robert,

Sorry for the late reply. The codes of CreateGeometryNode looks like this: 

osg::ref_ptr<osg::MatrixTransform> CreateGeometryNode() {

    std::vector<float> xyzs = RandomPoints(); ///< Create ramdon number of points 

    osg::Vec3Array vs = new osg::Vec3Array(xyzs.size()/3, (const osg::Vec3 *)(xyzs.data()));

    osg::ref_ptr<osg::Geometry> geo = new osg::Geometry;

    geo->setVertexArray(vs.get());

    geo->setColorArray(osg::Vec3(0.5,0.5,0.5));

    geo->setColorBinding(osg::Geometry::BIND_OVERALL);

    geo->addPrimitiveSet(new osg::DrawArrays(osg::PrimitiveSet::POINTS, 0, numpt));

    geo->getOrCreateStateSet()->setMode(GL_LIGHTING, osg::StateAttribute::OFF);

    geo->getOrCreateStateSet()->setAttribute(new osg::Point(size), osg::StateAttribute::ON);

    osg::ref_ptr<osg::MatrixTransform> mt = new osg::MatrixTransform;

    osg::ref_ptr<osg::Geode> geode = new osg::Geode;

    geode->addDrawable(geo.get());

    mt->addChild(geode);

    return mt; 

}

There's an old post from you which said that OSG will cache objects to improve performance, I wonder if this is the case and how I could change that behavior. 

Thanks! 

[Hui Li]

Here in my case, the memory growth is not unbounded,it keeps growing until ~5GB (described in the original post).  And the link I mentioned before is: https://groups.google.com/forum/#!topic/osg-users/mp_g_TbmAcY

[Robert Osfield]

Hi Hui,

On Monday, 1 June 2020 02:28:04 UTC+1, Hui Li wrote:

Hi Robert,

Sorry for the late reply. The codes of CreateGeometryNode looks like this: 

osg::ref_ptr<osg::MatrixTransform> CreateGeometryNode() {

    std::vector<float> xyzs = RandomPoints(); ///< Create ramdon number of points 

    osg::Vec3Array vs = new osg::Vec3Array(xyzs.size()/3, (const osg::Vec3 *)(xyzs.data()));

    osg::ref_ptr<osg::Geometry> geo = new osg::Geometry;

    geo->setVertexArray(vs.get());

    geo->setColorArray(osg::Vec3(0.5,0.5,0.5));

    geo->setColorBinding(osg::Geometry::BIND_OVERALL);

    geo->addPrimitiveSet(new osg::DrawArrays(osg::PrimitiveSet::POINTS, 0, numpt));

    geo->getOrCreateStateSet()->setMode(GL_LIGHTING, osg::StateAttribute::OFF);

    geo->getOrCreateStateSet()->setAttribute(new osg::Point(size), osg::StateAttribute::ON);

    osg::ref_ptr<osg::MatrixTransform> mt = new osg::MatrixTransform;

    osg::ref_ptr<osg::Geode> geode = new osg::Geode;

    geode->addDrawable(geo.get());

    mt->addChild(geode);

    return mt; 

}

Adding and removing data continuously is least efficient way to implement this type of functionality, it's not just an OpenGL/OSG thing, it applies pretty well to all computing.

If you have a stream of data coming in then the best way to manage it is to allocated fixed size data and then fill it in dynamically.  So no adding and removing, no new and delete on every frame.

If you have a really large dataset then chunk the data into blocks, 10,000 vertices per osg::Gemetry in a good place to start, but make this variable so you can figure what is the sweet spot for your system.

To vary th enumber of points draw using single DrawArrays per osg::Geometry and simply change the count to match the number of vertices you want to render.

If your data stream suddenly needs more points to represent it and your don't have enough available in your bricks then that is the time to allocate new bricks.

As for the cache, well the OSG does have cache for reusing GL objects but it intended to be used for the types of usage you are pushing it through.  Follow the above steps and your application will work perfectly.

Cheers,

Robert.