[osg-users] Ideas about a resource system, and deferred rendering framework

[Wang Rui]

Hi Robert, hi all,

During the past few days, I was thinking of implementing a resource system, as well as a rendering structure which could support both forward and deferred rendering for OSG. This could help developers design complex materials and effects, test cutting-edge GPU techniques, and implement multi-pass / deferred rendering pipelines with user-friendly interfaces or even a readable script format. Similar work were already done in some game engines like Ogre3D, CryEngine and Unreal, and I now plan to work out something with the same power and can fit our OSG users much better. :-)

My initial thoughts are:

1. A resource system records all the resource used in the rendering pipeline, including textures, state attributes, shaders, uniforms, camera parameters, and semantics (not used in GLSL, but very useful IMHO). Resource can be referred by other resource, or obtained from APIs to be altered. It can be recorded in an XML-based format for reading/writing, and external uses.

2. A rendering framework uses one or more techniques, passes, and connect their inputs/outputs for complex render work. It can also be recorded by the resource system and is implemented as a group node in the scene graph, which performs actual forward/deferred rendering work.

3. Some in-built techniques and passes can help implement some complex effects on customized scene quickly. Common ones I planned include HDR, SSDO, godrays, depth of field, motion blur, lensflare, color grading and FXAA. A benchmark could be developed first to show how the framework works.

4. Reader/writers could be developed then to convert DAE, CGFX, or even other game engine scripts into OSG native rendering pipelines, which will greatly help migrations from other engines.

The resource system itself is actually abstract and can be extended to contain whole scene information later, so it could be placed in the osgDB library. And the new rendering pipeline, which can totally replace current osgFX functionlities, can be placed in osgFX and rendering resource management will be done here, too.

I'm glad to work with others who has interests in such an idea, and hope an initial version could be finished before OSG 3.2. For the first stage I will borrow some code from the osgPostEffects library in my experimental osgXI project, to quickly build the low-level APIs of the framework. But anyone who have better ideas, or could contribute some code or effects will be really appreciated.

Cheers,

Wang Rui

[Robert Osfield]

Hi Rui,

Great timing. I've been wondering about the value of developing a
high level scene effects library for the OSG that would sit between
the viewer and model scene graph to enable effects such as you've
mentioned that take advantage of viewer level threading as well
integrate with existing OSG effects like libraries like osgShadow. I
haven't got to the point of spec'ing up what form this library might
take, let alone getting down to the design and prototyping stage so I
suspect you'll be further along than myself on it so look forward to
hearing more about what you are proposing/working on.

Getting these feature in for OSG-3.2 might be possible but personally
I'd like to get 3.2 sooner rather than later so would suggest aiming
for after 3.2. However, as much as I wanted to get 3.2 out already my
work and personal life have rather swept me off my feet in the last
six months so getting on to 3.2 has taken me rather longer than I ever
wanted... Over the next week I'll be taking stoke of all my
outstanding work and what is planned for the next few months so will
have a think about 3.2 as part of this. I can't make a 3.2 without
the community so it's something will need to fit in with availability
of others as well. For now I'd suggest just concentrate on your
proposals and if they fit into 3.2 time frame than all good, if not no
worry, we can just aim for 3.4.

Cheers,
Robert.

> _______________________________________________
> osg-users mailing list
> osg-...@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>
_______________________________________________
osg-users mailing list
osg-...@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[webmaster]

Great and valuable design and works!

   06,21,2012

在2012-06-21，"Wang Rui" <wang...@gmail.com> 写道:

-----原始邮件-----
发件人: "Wang Rui" <wang...@gmail.com>
发送时间: 2012年6月21日 星期四
收件人: "OpenSceneGraph Users" <osg-...@lists.openscenegraph.org>
主题: [osg-users] Ideas about a resource system, and deferred rendering framework

[Thrall, Bryan]

Wang Rui wrote on 2012-06-21:
> 2. A rendering framework uses one or more techniques, passes, and
connect
> their inputs/outputs for complex render work. It can also be recorded
by the
> resource system and is implemented as a group node in the scene graph,
> which performs actual forward/deferred rendering work.

It seems like osgPPU already does this; you might want to look at it to
see if it meets your needs, or at least could inspire what you're doing.

Hope this helps!
--
Bryan Thrall
Principal Software Engineer
FlightSafety International
bryan....@flightsafety.com

[Wang Rui]

Thanks Bryan,

Yes, I paid close attention to osgPPU since it appeared. I believe it would be very helpful for me to design the framework, although the script system we are going to use may differ. :-)

Wang Rui

2012/6/21 Thrall, Bryan <bryan....@flightsafety.com>

[Peter Bear]

I've been doing some thought of this kind of stuff myself, and it was actually a part of the reason I was considering hopping over to Ogre3D. I would actually suggest however taking a look at Light Indexed Deferred Rendering or Inferred Rendering as possible alternate implementations however, which address some of the issues deferred rendering has at the expense of a little bit of performance. There is something else to take in to account performance wise. It may be wise to design the resource manager in a data-orientated way, meaning to store all of a variable's values, such as a vec3's x value, in a single array. dtEntity's properties class seems to do this. This will assist in preventing cache misses, and thus increase the speeds of going through data. I would suggest however using memsize types where possible (I.E. using size_t instead of unsigned int), which will eat up slightly more memory, but will also help prevent memory allocation errors on 64 bit platforms.

Cheers,
Peter

------------------
Read this topic online here:
http://forum.openscenegraph.org/viewtopic.php?p=48415#48415

[Sergey Kurdakov]

Hi Wang,


might be not quite a significant point,
but osg based Flightgear has a ongoing Project Rembrandt

http://wiki.flightgear.org/Project_Rembrandt

it has some similar things done,
so could serve for inspiration.

Regards
Sergey

[Frederic Bouvier]

I was going to raise that point. Flightgear has for some years now a system where designers can express their statesets in XML files called effects. A statesets is called a "Pass" in FG (or a FG Pass is implemented with a stateset). You can add multiple passes in a Technique, allowing you to redraw the same geometry with multiple statesets (including render bins details). An effect file gathers multiple techniques that have a predicate, based on supported extensions and internal FG switches, which is evaluated at run time. The multiple techniques allow us to render the water or the urban areas differently based on user preferences and GPU capabilities.

Beside that effect system, we introduced a deferred renderer called 'Rembrandt' (currently still experimental but it will be present in the next release in August) where the different stages of the renderer are also described in XML. In that XML we can add rendering buffers (textures for RTT), specify rendering orders and assign effects to stages. The default pipeline has boom and ambient occlusion that can be switched on and off at run time.

If you have any question, you can consult the wiki page or ask

Regards,
-Fred


----- Mail original -----
De: Sergey Kurdakov

[Wang Rui]

Hi Fred and Sergey,

Thanks for the great information. I viewed the Rembrandt wiki page and love the design of the effect files. I will consider merge some of the concepts of Rembrandt if possible, along with current osgPPU and osgPostEffects projects, to try designing an extensible system for all. I would like to submit some of my initial code here for all of you to review and see if it could fit everyone's taste. I hope I could have done that in the following week.

Thanks,

Wang Rui

2012/6/22 Frederic Bouvier <fredl...@free.fr>

[Wang Rui]

Hi all,

Just to announce that the material system is still in progress and
I've just made the first version for myself to test and use. An
optimized version with a complete deferred shading pipeline (XML
files) will be submitted if it could fit most requirements.

As to share my current work with anyone who are also interested in
this idea and implementations, I'd like to attach the source files
along with this mail instead of submitting it in full fling. If you do
have better ideas, think the work is misdirected or needs to be fixed
in some parts, please don't hesitate to tell me. A second version with
more test files and a detailed XML file introduction will be updated
soon after some days.

Thanks,

Wang Rui


2012/6/21 Wang Rui <wang...@gmail.com>:

[Jeremy Moles]

On Mon, 2012-09-10 at 22:57 +0800, Wang Rui wrote:
> Hi all,
>
> Just to announce that the material system is still in progress and
> I've just made the first version for myself to test and use. An
> optimized version with a complete deferred shading pipeline (XML
> files) will be submitted if it could fit most requirements.
>
> As to share my current work with anyone who are also interested in
> this idea and implementations, I'd like to attach the source files
> along with this mail instead of submitting it in full fling. If you do
> have better ideas, think the work is misdirected or needs to be fixed
> in some parts, please don't hesitate to tell me. A second version with
> more test files and a detailed XML file introduction will be updated
> soon after some days.
>
> Thanks,

I'm testing this out, but you appear to have left noise_tex.jpg out of
the zip. :)

[Frederic Bouvier]

Hi Jeremy,

De: "Jeremy Moles"
>
> I'm testing this out, but you appear to have left noise_tex.jpg out of
> the zip. :)

You should be able to use this one :
https://gitorious.org/fg/fgdata/blobs/master/Textures/noise_tex.jpg
;-)

Regards,
-Fred

[Jeremy Moles]

On Mon, 2012-09-10 at 22:57 +0800, Wang Rui wrote:

> Hi all,
>
> Just to announce that the material system is still in progress and
> I've just made the first version for myself to test and use. An
> optimized version with a complete deferred shading pipeline (XML
> files) will be submitted if it could fit most requirements.
>
> As to share my current work with anyone who are also interested in
> this idea and implementations, I'd like to attach the source files
> along with this mail instead of submitting it in full fling. If you do
> have better ideas, think the work is misdirected or needs to be fixed
> in some parts, please don't hesitate to tell me. A second version with
> more test files and a detailed XML file introduction will be updated
> soon after some days.

This looks really cool so far. I'd be really interested to know if it
supports the following (and would be willing to create examples if
you're willing to help)...

Scenario: I want to render an entire subgraph to an FBO texture once,
then apply 2 or more completely different shaders in some order, then
put the final result into a last texture to be used somewhere in the
scene. I'm doing a guassian blur, which typically applies two different
shaders for x and y.

I have this working in osgPPU, but I think you already have enough to do
it here, I just couldn't put the pieces together. :)

[Wang Rui]

Hi Jeremy,

Thanks for the tests and feedback. I'm focusing on creating a material
system which may be a little similar to the Ogre one but will be very
easy to integrate with OSG scenes. I'd like to also have a benchmark
including a complete deferred shading pipeline in the near future to
show others how OSG produces realistic worlds. :-)

Your requirement could be easiliy implemented with one forward pass
rendering the scene to a texture, and two deferred passes doing the
blur work with the texture as input. A final compositing pass will
make use of the outputs of the blur passes and output to a new
texture. You can get and use the new texture then in the scene for
your own purpose instead of direct displaying them on screen. I'd like
to upload a DOF effect file and an updated example some days later to
demonstrate how these work.

Thanks,

Wang Rui

2012/9/11 Jeremy Moles <cubi...@gmail.com>:

> On Mon, 2012-09-10 at 22:57 +0800, Wang Rui wrote:
> This looks really cool so far. I'd be really interested to know if it
> supports the following (and would be willing to create examples if
> you're willing to help)...
>
> Scenario: I want to render an entire subgraph to an FBO texture once,
> then apply 2 or more completely different shaders in some order, then
> put the final result into a last texture to be used somewhere in the
> scene. I'm doing a guassian blur, which typically applies two different
> shaders for x and y.
>
> I have this working in osgPPU, but I think you already have enough to do
> it here, I just couldn't put the pieces together. :)
>

[Sergey Polischuk]

Hi

functionally it is about same as osgPPU as far as i can tell, but more straightforward and easy to use. I like it :)
Was missing ogre-like materials and compositiors when working with osg, tyvm for your work.

10.09.2012, 18:57, "Wang Rui" <wang...@gmail.com>:

> ,

[Jeremy Moles]

On Tue, 2012-09-11 at 10:54 +0800, Wang Rui wrote:
> Hi Jeremy,
>
> Thanks for the tests and feedback. I'm focusing on creating a material
> system which may be a little similar to the Ogre one but will be very
> easy to integrate with OSG scenes. I'd like to also have a benchmark
> including a complete deferred shading pipeline in the near future to
> show others how OSG produces realistic worlds. :-)
>
> Your requirement could be easiliy implemented with one forward pass
> rendering the scene to a texture, and two deferred passes doing the
> blur work with the texture as input. A final compositing pass will
> make use of the outputs of the blur passes and output to a new
> texture. You can get and use the new texture then in the scene for
> your own purpose instead of direct displaying them on screen. I'd like
> to upload a DOF effect file and an updated example some days later to
> demonstrate how these work.

Are there ever cases, when doing sophisticated layering of rendering
like this, that you'd want to manually "kick off" the EffectCompositor
for just a single frame and update the texture only once? (For example,
by setting the NodeMask to 0xF for one frame, then back to 0x0 when
you're done updating the View).

Is there a term for this kind of approach, and would it make sense to
also support this model of rendering directly or should it be left up to
the user?

[Jean-Sébastien Guay]

Hi Jeremy,

Is there a term for this kind of approach? Well "impostors" comes to
mind. That's where you would replace a complex mesh by a quad with a
texture representing the complex mesh. Then when some criteria changes
significantly (orientation of the view to the object, light source
position, etc) you would re-render the texture once to fit the new
criteria. This can be useful for far away meshes, for example trees
where a close mesh is used, then lower and lower LODs, and eventually
just a quad with a texture. Impostors are used so that the texture is
still dynamic, instead of just a static billboard (which tend to look
obviously like static billboards).

Hope this helps,

J-S

--
______________________________________________________
Jean-Sebastien Guay jean...@videotron.ca
http://whitestar02.dyndns-web.com/

[Wang Rui]

Hi Jeremy,

I will add a setEnable() method for each technique and pass of the
compositor so we can control the forward/deferred rendering work more
accurately. A texture which is attached with certain pass won't be
updated if the pass is disabled, and will become activated when the
pass is enabled again. I may upload the new version this weekend and
hope it will help achieve your goal.

Wang Rui

2012/9/11 Jeremy Moles <cubi...@gmail.com>:

> On Tue, 2012-09-11 at 10:54 +0800, Wang Rui wrote:
> Are there ever cases, when doing sophisticated layering of rendering
> like this, that you'd want to manually "kick off" the EffectCompositor
> for just a single frame and update the texture only once? (For example,
> by setting the NodeMask to 0xF for one frame, then back to 0x0 when
> you're done updating the View).
>
> Is there a term for this kind of approach, and would it make sense to
> also support this model of rendering directly or should it be left up to
> the user?
>

[Jeremy Moles]

On Tue, 2012-09-11 at 10:54 +0800, Wang Rui wrote:

> Hi Jeremy,
>
> Thanks for the tests and feedback. I'm focusing on creating a material
> system which may be a little similar to the Ogre one but will be very
> easy to integrate with OSG scenes. I'd like to also have a benchmark
> including a complete deferred shading pipeline in the near future to
> show others how OSG produces realistic worlds. :-)
>
> Your requirement could be easiliy implemented with one forward pass
> rendering the scene to a texture, and two deferred passes doing the
> blur work with the texture as input. A final compositing pass will
> make use of the outputs of the blur passes and output to a new
> texture. You can get and use the new texture then in the scene for
> your own purpose instead of direct displaying them on screen. I'd like
> to upload a DOF effect file and an updated example some days later to
> demonstrate how these work.

Hi Wang,

Did you ever get a chance to work up an example showing something like
this? I've been trying to get it to work using a modified blur-x/blur-y
approach from osgPPU, but have had no success.

[michael kapelko]

What's the progress on this? And is there any open repository?

2012/9/17 Jeremy Moles <cubi...@gmail.com>

[Wang Rui]

Hi Michael,

Current implementation is maintained in osgRecipes (https://github.com/xarray/osgRecipes), and some snapshots are shared in the mail-list a few days ago using both VDSM and the compositor.

Wang Rui

2012/11/17 michael kapelko <kor...@gmail.com>