Culling with glClipControl

[Chris Djali / AnyOldName3]

Hi,

glClipControl (available via osg::ClipControl) lets you specify how OpenGL converts between clip-space and window-space. This has the advantage that it lets you avoid the depth precision issues associated with the reverse-Z technique (described here https://nlguillemot.wordpress.com/2016/12/07/reversed-z-in-opengl/) when using OpenGL's default clip-space range of -1 to 1 instead of Direct3D/Vulkan/Metal's range of 0 to 1.

Obviously, to make use of this you need to make other changes, like using an alternative projection matrix, but mostly those are minor and well within the capabilities of the OSG.

There's one particular pain point, though. When the cull visitor applies a new camera, it calls CullStack::pushProjectionMatrix, which, among other things, sets up a new polytope for culling. This works by starting with a frustum representing clip space and transforming it using the projection matrix as the inverse in order to get one representing view space. The flaw with this is that it assumes the OpenGL default clip space is being used instead of whatever's been set with Clip Control.

For OpenMW, we've been experimenting with reverse Z and it's been ludicrously effective at removing Z-fighting with huge viewing distances. However, avoiding the culling issue described above has required a workaround that isn't as clean as we'd like.

I'm hoping that some discussion about how to potentially resolve the issue in the OSG itself will lead to a better long-term solution, and also inspire a better workaround we can use when targeting older OSG versions, too.

Thanks as always,

Chris

[Robert Osfield]

Hi Chris,

I think the only straight forward way to modify the CullStack::pushProjectionMatrix(..) operator would be to provide a default unit frustum polytope to use in place of the line (from CullStack::pushProjectionMatrix():

cullingSet.getFrustum().setToUnitFrustum(((_cullingMode&NEAR_PLANE_CULLING)!=0),((_cullingMode&FAR_PLANE_CULLING)!=0));

This would require modifications to the core OSG.

I can't think of any other way to do this, it really isn't something that the OSG was written to handle - the -1 to 1 unit clipping was something set in stone for decades...

Robert.

[Robert Osfield]

H Chris,

I realize this doesn't help you directly, bit out of curiosity I've looked up the equivalent class from tracking the projection/view matrix/view frustum and it use a vsg::State::_frustumUnit that is initialized to a -1 to 1 unit cube, but would potentially allow one to overwrite this. prior to the cull/record traversal.

Robert.