[osg-users] osgOcean call for testing and future

[Jean-Sébastien Guay]

Hello all,

I have not had any contact with Kim Bale, who had started and was
heading the osgOcean project, since at least a few months, despite a few
e-mails from me and questions from others in that time. His last
activity on osgOcean was in May of this year (see
http://code.google.com/p/osgocean/updates/list). I am committer on the
project and have made modifications with Kim's consent in the past, but
now I find I might have to "take the reins" in Kim's stead if I don't
hear from him in the near future.

First of all, would anyone have heard from Kim lately? I have no other
means to communicate with him than the e-mail address I was using all
this time, and I'm wondering if something may have happened on a more
personal level (other than just changing jobs/interests which surely may
happen). So if anyone has a way to reach Kim, just to let us know
everything's all right, I'd appreciate it.

Secondly, I have recently reintegrated some changes I'd made locally
over a few months into osgOcean trunk, and was considering making a
release soon in order to keep the project moving forward. If anyone is
using osgOcean 1.0.1 and would like to help me out a bit, could you just
download the SVN trunk and test it out in your apps? The URL is:

http://osgocean.googlecode.com/svn/trunk/

The changes I've made will let apps decide whether they use the inline
shader source or customized shader files they would distribute with
their apps, and also let apps decide when and how fast the ocean surface
updates itself. More details:

Shaders: Previously, osgOcean hard-coded a value for each class that
used shaders, called USE_LOCAL_SHADERS, and this value decided if
osgOcean used the inline shader source or loaded the shader from file.
If an app wanted to customize one of the ocean shaders, for example the
ocean surface shader, and it had compiled osgOcean with
USE_LOCAL_SHADERS=1 in the FFTOceanSurface.cpp file, then it either had
to traverse to find the ocean surface node and override the shader
there, or recompile osgOcean with USE_LOCAL_SHADERS=0. This was all too
complicated and made integration difficult.

Now, osgOcean combines both ways. It will first try to load the shader
from file (and all shader files have been prefixed with "osgOcean_" to
prevent it from loading the wrong one if the user had a shader called
"heightfield" for example...) and if the shader is not found in the OSG
data paths, then it will use the inline shader source. This is more
flexible, so apps can decide not to distribute any shader files if they
don't want to, and if they want to customize one shader they just have
to distribute that one and make sure the OSG data path is set to find it.

Eventually I'd like to let the user specify their shader filename or
source directly to have maximum flexibility, but I'll have to think
about how to design/expose this in a good way.

Ocean surface updating: We run osgOcean with viewports spanning screens
on multiple computers, and we have dynamic objects that follow the ocean
surface. Both these factors meant we needed to have a way to control the
ocean surface update from some external source. FFTOceanSurface now has
a public OceanUpdateCallback that users can override. Using this, we
broadcast a simulation time from one master to all graphics slaves, and
both the dynamics and graphics (including the ocean surface) use this
value to update themselves. This makes the ocean surface synchronized
everywhere.

There were also a few small bug fixes made, which will be in the release
notes when I decide to make a release.

So, anyone interested in doing a bit of testing? :-)

Finally, if I still can't get any sign from Kim in the next few weeks,
I'll go over the active discussions in the issue tracker on the
googlecode site and see if I can respond to or close any of them. I
wasn't checking that at all in the past, thinking Kim was much better
placed to respond (since most changes I've made were integration/class
design changes and not the actual rendering/shaders) but now I'll need
to get more involved. And another thing is the submission of river
rendering that Quintijn Hendrickx made a while back, I'll be looking to
integrate that in the near future.

Thanks,

J-S
--
______________________________________________________
Jean-Sebastien Guay jean-seba...@cm-labs.com
http://www.cm-labs.com/
http://whitestar02.webhop.org/
_______________________________________________
osg-users mailing list
osg-...@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[Jean-Sébastien Guay]

Hi all,

Just a quick update:

> First of all, would anyone have heard from Kim lately? I have no other
> means to communicate with him than the e-mail address I was using all
> this time, and I'm wondering if something may have happened on a more
> personal level (other than just changing jobs/interests which surely may
> happen). So if anyone has a way to reach Kim, just to let us know
> everything's all right, I'd appreciate it.

Nick sent me his phone number and I just spoke to Kim, seems I didn't
have to be concerned. He'll probably send us a message soon, but I
wonder if some erroneous e-mail filtering might be involved...

Sorry for sounding alarmed, seems all is OK.

Now that that's out of the way, if anyone is interested in testing
osgOcean trunk, we'll probably want to make a release in the near future.

[Kim Bale]

Hi J-S,

I am here, alive and well and I must apologise for my absence. It's lovely to know that I'm missed when I'm absent though so thanks for your concern. :) Firstly I'll explain my inactivity, over the past few months I got burnt on a couple of osgocean contract jobs and consequently became rather disenchanted with the whole open source thing and took a bit of a break from it all. However, I think it's time to get back on it. It's my baby and I don't want it to become another stale open source project. 

So to your changes. The change to the way the shaders are integrated is a good approach in fact, the shader integration has troubled me for a while. My goal with inlining the shader code was to keep the library as compact as possible. However, it soon became a maintenance nightmare - people were submitting shader fixes and only editing either the inline code or the external files and keeping track of which version was correct was becoming very difficult. 

I've already done some work to address this. So what I will do is augment your approach with a pre build script which will read the external shader files and generate headers which contain the shader code that are then included in the build. This should go some way to preventing any discrepancies between inline shaders and external shaders. I have already written batch scripts to do this in windows, and I think I made some progress in writing a similar bash script for linux too but I'm not sure whether I finished it. I'll have to have a dig around. Thinking about this, more recently I've starting using python as my scripting language of choice - I'd be keen to use python to do this but I'm not sure how people would feel about the additional build dependencies.. do you have any thoughts on this? Simply put it would make my life a bit easier since I'm windows based and testing bash scripts is a bit of a pain. 

The ocean update callback is also a good approach, although I'll have to make sure to document it on the wiki otherwise it'll be missed. I had implemented a method aimed at achieving the same thing a few months back but yours is more extensible so I'm happy to use that. 

Since my last update I have fixed the VBO ocean technique so I'll commit that. I was rather disappointed by the performance gain though I noted a 10% drop on the draw traversal and a 80% drop in the update but no significant change in the frame rate. It also has the side effect of creating strange normals on the low resolution tiles. I'm rather dubious as to whether I've implemented the VBO update correctly so it would be good to see if improvements could be made there. 

Lastly I have some minor changes to the example application, just some housework to make it easier to read and test the VBO technique. 

I'll make a push to get these committed over the weekend, and will of course give your recent commits a thorough testing. Given the amount of fixes that have come in since 1.0.1 I think osgocean is due for another release so I'll make these changes asap and focus on getting another release out.

Thanks again,

Kim.

[Jean-Sébastien Guay]

Hi Kim,

Great to know everything is all right. I understand one can get burnt
out on a particular project after a while, you didn't go into specifics
and I totally understand that it may be a delicate subject but I'm glad
to see you back at it once again! :-)

> So to your changes. The change to the way the shaders are integrated is
> a good approach in fact, the shader integration has troubled me for a
> while. My goal with inlining the shader code was to keep the library as
> compact as possible. However, it soon became a maintenance nightmare -
> people were submitting shader fixes and only editing either the inline
> code or the external files and keeping track of which version was
> correct was becoming very difficult.

I agree 100%.

> I've already done some work to address this. So what I will do is
> augment your approach with a pre build script which will read the
> external shader files and generate headers which contain the shader code
> that are then included in the build. This should go some way to
> preventing any discrepancies between inline shaders and external
> shaders. I have already written batch scripts to do this in windows, and
> I think I made some progress in writing a similar bash script for linux
> too but I'm not sure whether I finished it. I'll have to have a dig
> around. Thinking about this, more recently I've starting using python as
> my scripting language of choice - I'd be keen to use python to do this
> but I'm not sure how people would feel about the additional build
> dependencies.. do you have any thoughts on this? Simply put it would
> make my life a bit easier since I'm windows based and testing bash
> scripts is a bit of a pain.

I think the python route would be the best actually, most developers
will already have a python installation, and I would expect they would
only need to run the script if they change shaders, otherwise the C++
files would stay the same. I was thinking of doing something like that
myself. Up to you if you'd rather tackle it.

Instead of headers I would suggest one or more .cpp file(s) that would
define say const char* osgOcean_water_surface_fragment_source = "...";
and then the FFTOceanSurface.cpp file would refer to that symbol and it
would be resolved at link time. A header suggests it would be part of
some public interface, which is not the case here.

> The ocean update callback is also a good approach, although I'll have to
> make sure to document it on the wiki otherwise it'll be missed.

Good point, I was actually also thinking of doing a documentation pass
because I think we're missing a kind of high-level document that
explains a few use cases and what should work (in particular it would be
nice to explain how to use osgOcean with osgShadow). Once again I can do
this in the near future, before we do the next release.

> Since my last update I have fixed the VBO ocean technique so I'll commit
> that. I was rather disappointed by the performance gain though I noted a
> 10% drop on the draw traversal and a 80% drop in the update but no
> significant change in the frame rate. It also has the side effect of
> creating strange normals on the low resolution tiles. I'm rather dubious
> as to whether I've implemented the VBO update correctly so it would be
> good to see if improvements could be made there.

I'll have a look when you get to committing the changes. The drops are
good (I assume you mean drop in timings, not drop in performance :-) ),
and I'm sure they will be more significant for real apps (in particular
the 80% drop in update time is very welcome as the ocean will be more
usable in debug builds now).

> I'll make a push to get these committed over the weekend, and will of
> course give your recent commits a thorough testing. Given the amount of
> fixes that have come in since 1.0.1 I think osgocean is due for another
> release so I'll make these changes asap and focus on getting another
> release out.

Great to see we're on the same page. I think before we make a release
the following milestones should be met:

1. Give the VBO technique more testing with a focus on eliminating
artifacts. If the technique is a step forward in performance but it has
some significant artifacts then it will be less likely to be used in
production applications - the performance gains will likely be overlooked.

2. Clean up the shader code duplication with an automatic generation
scheme. Again python would be my choice.

3. Expand on the usage documentation on the wiki, and perhaps include
some documentation in the release tarball/zip as well.

On a separate subject, we promised to have a look at Quintijn
Hendrickx's osgRiver code to see if it would make sense for inclusion
into osgOcean. I suggest we tackle this after the release, but it might
be nice to have a look at that separately and then have a few
back-and-forths to discuss what we thought. I just don't want to forget
about it. I think I'll log an issue in the issue tracker for that.

Thanks for getting back in touch, and for helping me push this forward
once again,

J-S
--
______________________________________________________
Jean-Sebastien Guay jean-seba...@cm-labs.com
http://www.cm-labs.com/
http://whitestar02.webhop.org/

[Kim Bale]

Hi J-S,

I think the python route would be the best actually, most developers will already have a python installation, and I would expect they would only need to run the script if they change shaders, otherwise the C++ files would stay the same. I was thinking of doing something like that myself. Up to you if you'd rather tackle it.

I've just implemented this and will commit tomorrow once I've gone through it to make sure I haven't missed anything.

1. Give the VBO technique more testing with a focus on eliminating artifacts. If the technique is a step forward in performance but it has some significant artifacts then it will be less likely to be used in production applications - the performance gains will likely be overlooked.

I concur, I'll merge it in over the weekend and see what can be done. 

2. Clean up the shader code duplication with an automatic generation scheme. Again python would be my choice.

3. Expand on the usage documentation on the wiki, and perhaps include some documentation in the release tarball/zip as well.

Well 2 is done and 3 shouldn't take too long, I'll chip away at it in the evenings. 

On a separate subject, we promised to have a look at Quintijn Hendrickx's osgRiver code to see if it would make sense for inclusion into osgOcean.

Absolutely, I feel rather bad about this actually. It's a fairly substantial chunk of code and requires a fair amount of thought design wise. I kind of think it might be possible for to exist as a separate library, but with so many shared shader effects it seems silly to duplicate the code. Will definitely address this after this release.

K.

[Jean-Sébastien Guay]

Hi Kim,

You seem keen on working on things all by yourself, are you sure I can't
help, at least on documentation? In that past two days I went from
thinking I was going to take over the project entire (at least for the
next release or two) to you coming back and doing everything yourself!
It's kind of a dramatic change of direction for me, and I would have
thought you would welcome the help and delegate a bit of the 3 item list
I made in my previous message :-)

That said, it's good to see activity on the project :-)

J-S

> <mailto:jean-seba...@cm-labs.com>

> http://www.cm-labs.com/
> http://whitestar02.webhop.org/
> _______________________________________________
> osg-users mailing list
> osg-...@lists.openscenegraph.org

> <mailto:osg-...@lists.openscenegraph.org>
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[Martin Naylor]

Hi JS,

I am receiving the following error in trying cmake.
CMake Error in src/osgOcean/CMakeLists.txt:
Cannot find source file "osgOcean_gaussian1.vert". Tried extensions .c .C
.c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx
Gaussion2.vert is also missing from the svn?

Regards

Martin.

[Kim Bale]

Hi Martin,

Yeah this is my fault, I shouldn't have committed the file delete its part of an update that will be coming today.

I'll fix this at lunchtime. 

K.

[Martin Naylor]

Cheers Kim, I will give it another go later....

 

Martin.

[Kim Bale]

This should be fixed now.

Note: the new CMake build uses a Python 2.7 script as a prebuild routine.  

Regards,

Kim.

[Kim Bale]

Hi J-S,

You seem keen on working on things all by yourself, are you sure I can't help, at least on documentation?

Haha yes of course you can help with the documentation. I only jumped in the first two since I'd already started them. 

I haven't really given the documentation much thought yet, but if you can identify some good cases to document go for it.

It might also be a good idea to cast your eye over the new scripting stuff I just committed, you may think of a better way of doing it.

Regards,

Kim.

[Martin Naylor]

HI Kim,

Yeah that done the trick, another quick question, where do I get islands.ive from?

 

Cheers

[Kim Bale]

Download the resources rar from here:

http://code.google.com/p/osgocean/downloads/list

K.