Usage of osg::BufferObjects

[Werner Modenbach]

Hi users,

I'm currently experimenting with the usage of meshlets in OSG.
Robert already implemented the corresponding draw methods. Thanks Robert!
The main thing necessary to make it work is the indexed access to almost every data given by the CPU,
which is very limited in the traditional shader pipeline. I.e. a vertex shader can only access 1 Vertex at a time.
Mesh shaders can fetch any vertex by index and any number of vertices.

A typical use/definition of the shader structures may be:

//-------------------------------------
// transform_ub: Uniform buffer for transformations
//
layout (std140, binding = 0) uniform uniforms_t { 
  mat4 ViewProjectionMatrix;
  mat4 ModelMatrix;
} transform_ub;
 
//-------------------------------------
// vb: storage buffer for vertices.
//
struct s_vertex {
	vec4 position;
	vec4 color;
};
 
layout (std430, binding = 1) buffer _vertices {
	s_vertex vertices[];
} vb;
 
 
//-------------------------------------
// mbuf: storage buffer for meshlets.
//
struct s_meshlet {
	uint vertices[64];
	uint indices[378]; // up to 126 triangles
	uint vertex_count;
	uint index_count;
};
 
layout (std430, binding = 2) buffer _meshlets {
	s_meshlet meshlets[];
} mbuf;

So all data is given via BufferObjects.

Meshlet shaders are kind of compute shaders with output to the rasterizer and thus a fragment shader.
The indexed access is like:

  uint mi = gl_WorkGroupID.x;
  uint thread_id = gl_LocalInvocationID.x;
 
  uint vertex_count = mbuf.meshlets[mi].vertex_count;
  for (uint i = 0; i < vertex_count; ++i) {
    uint vi = mbuf.meshlets[mi].vertices[i];
 

Precondition for the indexed access is having the data given via BufferObjects.
In OSG we have only predefined types (Indices, Vertices and Images) for BufferObjects.

Has anybody any experience deriving a more general class for putting arbitrary structs in BufferObjects (i.e. struct s_meshlet)?
I think that shouldn't be such a big deal or am I overseeing something.

@Robert: Or can you provide any hints to me for creating such a derived class?

If I manage getting this working I'll provide it to the community because mesh shaders are really great for many applications!

Many thanks in advance

- Werner -

[DC]

I'm very curious if you have had any luck getting this to work in OSG.

After watching some of the online demo's regarding mesh shaders (specifically Unreal), it would be great if there was a way for this to be implemented in OSG.

[Werner Modenbach]

Hi DC,
it took me about one year, fully understanding task and mesh shader.
I converted all my software to this technology and I'm fascinated by it's functionality and performance. At what point did you stock?
Werner

[DC]

Hi Werner,

Have you been able to implement this in OSG?

I just started looking at mesh shaders and found your post. I'll need to learn how to generically break up a model into meshlets.

Did you find any good tutorials that do more than just draw a triangle?

Thanks,

DC

[Werner Modenbach]

Hi DC,
I think we should clarify some aspects before going into the details. If I understand well, your problem can be split into 3 parts
1) You want to convert an existing model into meshlets. This is not related to OSG. But I found a git repository from NVIDIA doing this job. I will lookup the link and send it to you tomorrow.
2) For this to work you need task/mesh shaders doing their job on the GPU. This is also not related to OSG. Maybe you find some starting point in the repository as well.
3) You need to know how to load your shader and how to setup the data structures in OSG. I hope I can help you with that by providing some code snippets.
Just let me know when you are ready for that step. But the code I provided already should help you.
Werner

[DC]

OK.  Thank you.

[Werner Modenbach]

Hi DC,

here is the link: https://github.com/nvpro-samples/gl_vk_meshlet_cadscene
This should help you with items 1) and 2)

For item 3) see my code below.
The way I assign the data structures in OSG is like that:

    // ----------------------
    // Create Meshlet primitive
    // ----------------------

    _geode->getOrCreateStateSet()->getOrCreateUniform(TASK_INVOCATIONS_PER_MESHLET, osg::Uniform::INT)->set(taskInvocationsPerMeshlet);

    osg::ref_ptr<osg::Geometry> geometry = new osg::Geometry;

    osg::ref_ptr<osg::ProxyPrimitive> proxyPrimitive = new osg::ProxyPrimitive;

    proxyPrimitive->drawMeshTasks = new osg::DrawMeshTasks(0, workgroupsInGeometry * taskInvocationsPerMeshlet);

    geometry->setUseDisplayList(false);

    geometry->addPrimitiveSet(proxyPrimitive.get());

    _geode->addChild(geometry);

    // ----------------------
    // Vertices

    // ----------------------

    osg::ref_ptr<osg::Vec4Array> osgVertices = new osg::Vec4Array(toCopy);

    for (auto i=0; i<toCopy; ++i) {    // fill array
        // Fill buffer
    }

    // Create Buffer and Texture

    osg::ref_ptr<osg::VertexBufferObject> verticesBufferObject = new osg::VertexBufferObject();

    verticesBufferObject->setUsage(GL_DYNAMIC_READ_ARB);

    osgVertices->setBufferObject(verticesBufferObject.get());

    osg::ref_ptr<osg::TextureBuffer> vertexTexture = new osg::TextureBuffer(osgVertices.get());

    vertexTexture->setInternalFormat(GL_RGBA32F_ARB);

    vertexTexture->setInternalFormatMode(Texture::USE_USER_DEFINED_FORMAT);

    stateSet->setTextureAttributeAndModes(MESH_VERTEX_ARRAY_TEXTURE_UNIT, vertexTexture.get());

    // ----------------------
    // MeshletInfos

    // ----------------------

    osg::ref_ptr<Vec4uiArray> MeshletInfosArray = new Vec4uiArray(workgroupsInGeometry);

    int startVertex = 1;

    int anzahl = _anzahlVertices;

    for (auto i=0; i<workgroupsInGeometry; ++i) {

        (*MeshletInfosArray)[i][0] = startVertex;

        (*MeshletInfosArray)[i][1] = min(TS_WORKGROUP_SIZE, anzahl);

        startVertex += TS_WORKGROUP_STEP;

        anzahl -= TS_WORKGROUP_STEP;

    }

    // Create Buffer and Texture

    osg::ref_ptr<osg::VertexBufferObject> meshInfosBufferObject = new osg::VertexBufferObject();

    meshInfosBufferObject->setUsage(GL_DYNAMIC_READ_ARB);

    MeshletInfosArray->setBufferObject(meshInfosBufferObject.get());

    osg::ref_ptr<osg::TextureBuffer> meshInfosTexture = new osg::TextureBuffer(MeshletInfosArray.get());

    meshInfosTexture->setInternalFormat(GL_RGBA32UI_EXT);

    meshInfosTexture->setInternalFormatMode(Texture::USE_USER_DEFINED_FORMAT);

    stateSet->setTextureAttributeAndModes(MESH_INFOS_ARRAY_TEXTURE_UNIT, meshInfosTexture.get());

    // ----------------------
    // Create and assign the Task- and Mesh shader as usual by using the corresponding shader type
    // ----------------------

[DC]

Thanks Werner!

This will certainly help me get started.

Much appreciated,

DC

[Trajce Nikolov NICK]

Hi,

I am also researching this subject. Have you succeeded? Any hints about the meshelts buffer objects? Any other hints?

Thank you a bunch for any shared info!

Nick

--
You received this message because you are subscribed to the Google Groups "OpenSceneGraph Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osg-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/7680959c-297e-41ff-9159-464b38c8df73n%40googlegroups.com.

--

trajce nikolov nick

[Werner Modenbach]

Hi Nick,
There is nothing special in the contents of the buffers. They are just arrays like vertex array. Special is the definition as buffer objects and "overlaying" them with a texture like shown in my code. In the shader you have the instance ID and from that you access the meshlet info via the 1D texture. Get your operation parameters from there and fetch vertices, normals, colors etc. also via the textures.
The only difficulty is the size limit of the textures. So you eventually have to split the arrays.
That's it.
Werner

[Trajce Nikolov NICK]

Hi!

Thanks a bunch! Is your code in the repo so I can have a look?

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/eb4531f5-3420-4c75-be46-539490c5c659%40email.android.com.

[Trajce Nikolov NICK]

Hi!

I haven't looked at the code yet, just want to clarify one thing: you are providing the meshlets verts uvs normals through 1d texture?

[Werner Modenbach]

Yes, that is the only (?) way.

[Trajce Nikolov NICK]

Okay. Thanks a buch for these  hints! I am starting next week with this. Will post findings

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/3fd7dcac-1cec-43d9-ac91-04e22f331bd9%40email.android.com.

[Trajce Nikolov NICK]

Hi Werner!

Thanks soo much for the shared code and the hints!!! I have a better understanding now how all works together wrt. buffers. Just this snippet, if you are willing to provide more hints.

int startVertex = 1;

    int anzahl = _anzahlVertices;

    for (auto i=0; i<workgroupsInGeometry; ++i) {

        (*MeshletInfosArray)[i][0] = startVertex;

        (*MeshletInfosArray)[i][1] = min(TS_WORKGROUP_SIZE, anzahl);

        startVertex += TS_WORKGROUP_STEP;

        anzahl -= TS_WORKGROUP_STEP;

    }

The ideal is to have a working code, a bit more robust then the one I found in the repo in the meshshaders branches. But I am getting there slowly :-)

This is indeed a big thing!

Big thanks!

Nick

Virus-free. www.avast.com

--

trajce nikolov nick

Virus-free. www.avast.com

[Trajce Nikolov NICK]

Hi again Werner,

I think I got it after reading few articles, actually the main one from NVIDIA. Do you have experience in using this with paged data?

--

trajce nikolov nick

[Werner Modenbach]

Hi Nick,
good to hear you are getting closer.
Due to the fact that standard classes of paged data handle vertex arrays etc. autonomously and mesh shaders work different I think you have to look up where this handling is done and write your own derived class.
Werner

[Trajce Nikolov NICK]

Yea, but I have a handle on that part. Can ve done via read callback in the registry, or saved as well, I saw there is serialization support. 

I wrote a simple visitor that should convert a loaded model into meshlets. I had it working nicely until I started to experiment with threaded access.

Attached is the source. Not elegant at all but I would be happy if you can have a look and maybe spot the problem

Thanks a bunch!

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/0ab4c82f-1f17-47e9-aa1d-9c02b5a3f7fb%40email.android.com.

[Werner Modenbach]

Hi Nick,
I think this is really the point where it is leaving meshlet specific support.
This is the standard developers work.
Werner

[Trajce Nikolov NICK]

Ya, you're right 😀. I made it work with paged terrain data. 

Any hints for optimal settings?

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/8dcaed4f-0992-482a-abdb-7d30e7f83b5b%40email.android.com.

[Werner Modenbach]

Hmmm...
This is a very global question.
Read the articles about mesh shaders and how to tailor them for optimal hardware usage.

[Trajce Nikolov NICK]

This is how it looks like. Thanks so much for the hints, it took like 12 hours to get there. Not significant performance boost though. Have to learn more. Thanks again!

Nick

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/f9c21199-c94b-4c2b-9c10-90f80ce12a57%40email.android.com.

[Werner Modenbach]

It took me a year for discovering all the wrong path I took and understanding how it works. But it was worth the effort. Performance factor 10 is my result.

[Trajce Nikolov NICK]

Yes, it is tricky. Also want to ask, how you are passing uvs,  via vertex attrs?

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/28f8812e-8ecb-4bfa-bd50-03a656a54ab8%40email.android.com.

[Werner Modenbach]

Hi Nick,
you really have to read the docs of mesh shading. There are NO predefined data structures. No vertices, no colors - just NOTHING! All data has to be packed into arrays and overlayed by textures.
Please go ahead yourself now. I cannot hold your hands during the development. Reading docs and studying the existing git repository is mandatory now.
Good luck
Werner

[Trajce Nikolov NICK]

Hi! I just wanted to clarify this, was expecting it that way. Anyway,  thank you for the support and the snippets! Couldn't do it that fast without your hints! I will soon share the code for others that might be interested in. This is what is missing here, a workable use case covering at least a bit more.if not all.

Thanks a bunch again!

Cheers,

Nick

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/de29377d-a7ae-418a-ab5b-388afb8a9f51%40email.android.com.

[Trajce Nikolov NICK]

Hi!

Here is a working code based on the discussion in thus thread. It is just using meshlets,  no performance boost, so we will study it further and if anyone else has suggestions, all are welcome! It was tested on large paged terrain. The attached image is a reference. 

Thanks a bunch again!

  osgmeshshader.cpp

[Werner Modenbach]

Hi Nick,

just converting something to meshlets doesn't make much sense.
The key aspect of meshlets is the optimisation of shader topology and shaders itself
for optimized usage of existing hardware, paralellism of kernels on the boards(warps), threading
model on the boards, thread cooperative functions, ultra high performance of shared memory
and optimizations possible due to free access to any user data.
All this, together with the abilitiy of task and mesh shaders in on board
dynamic generation of mesh shader instances, enables you to do on board LOD handling,
on boerd culling and much more. Here is the real potential for gaining performance boosts.
So you have to learn about the hardware and how to use it in an optimal way.

Werner

To view this discussion on the web visit https://groups.google.com/d/msgid/osg-users/CAO-%2Bzime2HmR_83cC8oR-i%2BAXZiwQwE3eKYqyMiOGLfywQvZgw%40mail.gmail.com.

[Trajce Nikolov NICK]

Thanks Werner! At least we have started with this :-). Indeed it is a lot to learn

--

trajce nikolov nick