[osg-users] Projective Multi-Texturing

[Christoph Heindl]

Dear list members and OSG - Team,

thank you for all the past and current efforts put into OpenSceneGraph! All of my colleagues (including myself) enjoy using your product on a daily basis. 

Currently, we are working on supporting texturing for product called ReconstructMe (real-time 3d reconstruction). Basically what we'd like to achieve is to use photos to texture a mesh. Our current state of development allows us to generate UV coordinates for a single triangle mesh for each photo (incorporates estimating camera position relative to mesh, determining visible triangles, etc.). 

What we'd like to do next is to visualize the textured mesh. Since we are all computer vision guys, we lack experience in computer graphics. Before heading in the wrong direction, I'd wanted to discuss possible ways to implement the visualization step with experts on this list.

After a bit of research I stumbled upon the following approaches to multi-texturing

 - using ARB_multitexture extension. Seems to be limited by available hardware units.

 - using GPU multitexturing using separate texture units. Also limited by available texture units on hardware

 - using multi-pass rendering. Probably slower but not limited by hardware.

Question 1: Is there a way to generically supply the textures and UV coordinate sets and let OSG choose the best rendering technique from the list above?

Question 2: If not, how would I go about list item number 2 and 3? Are there any samples around?

Question 3: Since each photo covers only a portion of the mesh, I guess we need to set texture coordinates for non-textured vertices to be outside of the valid UV range (e.g. -1) and set a black-transparent border color? How would one combine the textures in a such a way that either only the first valid texture is applied to the mesh for any given triangle?

Thanks in advance,

Christoph

[Christoph Heindl]

--- Note this message has been sent to the list, but somehow does not show up..so this is a resend.

[Jason Daly]

On 10/23/2012 02:35 AM, Christoph Heindl wrote:

After a bit of research I stumbled upon the following approaches to multi-texturing

 - using ARB_multitexture extension. Seems to be limited by available hardware units.

 - using GPU multitexturing using separate texture units. Also limited by available texture units on hardware

There's no difference between these two.  ARB_multitexture is a 15-year old extension that simply provides the specification for how multitexturing is done.  Multitexturing has been part of standard OpenGL since version 1.3.

 - using multi-pass rendering. Probably slower but not limited by hardware.

I doubt you'll need to resort to this, but with the vague description of what you're doing, I can't be 100% sure.

Question 1: Is there a way to generically supply the textures and UV coordinate sets and let OSG choose the best rendering technique from the list above?

Again, I can't be sure what you're trying to do from your brief description, but it sounds like you already have a mesh generated from photos and you now just want to project those photos (which have a known location, orientation, etc) onto the mesh.

If this is true, you can generate the texture coordinates directly from the position/orientation of the photo and the positions of each vertex in the mesh.  Read up on projective texturing to see how this is done.  OSG can do this easily if you write a shader for it, or you can use the osg::TexGen state attribute to handle it for you (it works just like glTexGen in regular OpenGL).

The other thing you'll need to do is divide up the mesh so that only the faces that are covered by a given photo or photos are being drawn and textured by those photos.  This will eliminate the need for you to have as many texture units as there are photos.  There may be regions of the mesh where you want to blend two or more photos together, and this is the only time where you'd need multitexturing.  You should be able to handle this mesh segmentation with a not-too-complicated preprocessing step.

Hope this helps,

--"J"

[Christoph Heindl]

Hi Jason,

On Tue, Oct 23, 2012 at 5:27 PM, Jason Daly <jd...@ist.ucf.edu> wrote:

There's no difference between these two.  ARB_multitexture is a 15-year old extension that simply provides the specification for how multitexturing is done.  Multitexturing has been part of standard OpenGL since version 1.3.

Ok I see, I stumbled upon this terms when looking at 

http://updraft.github.com/osgearth-doc/html/classosgEarth_1_1TextureCompositor.html

 - using multi-pass rendering. Probably slower but not limited by hardware.

I doubt you'll need to resort to this, but with the vague description of what you're doing, I can't be 100% sure.

Actually what I do is that I have a mesh that is generated from depth-maps. In a post-processing step I want to apply photos (taken by arbitrary cameras, but with known intrinsics) as textures. What I know is the position of which the photo was taken (relativ to the mesh) and the camera intrinsics.

What I do next is to calculate UV coordinates myself using projective texturing. Of course, not all triangles of the mesh get textured by the same photo, and there a triangles that are not visible from any photo.

If this is true, you can generate the texture coordinates directly from the position/orientation of the photo and the positions of each vertex in the mesh.  Read up on projective texturing to see how this is done.  OSG can do this easily if you write a shader for it, or you can use the osg::TexGen state attribute to handle it for you (it works just like glTexGen in regular OpenGL).

How can TexGen and a shader help here? Would it allow me to calculate the UV coordinates for a given photo (camera position etc.) and the mesh?

 

The other thing you'll need to do is divide up the mesh so that only the faces that are covered by a given photo or photos are being drawn and textured by those photos.  This will eliminate the need for you to have as many texture units as there are photos.  There may be regions of the mesh where you want to blend two or more photos together, and this is the only time where you'd need multitexturing.  You should be able to handle this mesh segmentation with a not-too-complicated preprocessing step.

I wanted to avoid splitting the mesh, at least for the internal representation (which I hoped included visualization). Pros and cons have been discussed in this thread (in case you are interested)

https://groups.google.com/d/topic/reconstructme/sDb_A-n6_A0/discussion

Best,

Christoph

[Jason Daly]

On 10/23/2012 11:59 AM, Christoph Heindl wrote:

Hi Jason,

On Tue, Oct 23, 2012 at 5:27 PM, Jason Daly <jd...@ist.ucf.edu> wrote:

There's no difference between these two.  ARB_multitexture is a 15-year old extension that simply provides the specification for how multitexturing is done.  Multitexturing has been part of standard OpenGL since version 1.3.

Ok I see, I stumbled upon this terms when looking at 

http://updraft.github.com/osgearth-doc/html/classosgEarth_1_1TextureCompositor.html

 - using multi-pass rendering. Probably slower but not limited by hardware.

I doubt you'll need to resort to this, but with the vague description of what you're doing, I can't be 100% sure.

Actually what I do is that I have a mesh that is generated from depth-maps. In a post-processing step I want to apply photos (taken by arbitrary cameras, but with known intrinsics) as textures. What I know is the position of which the photo was taken (relativ to the mesh) and the camera intrinsics.

OK, that makes sense.  It doesn't change what I said earlier, you can still do this with projective texturing.

How can TexGen and a shader help here? Would it allow me to calculate the UV coordinates for a given photo (camera position etc.) and the mesh?

The more I think about it, the more I think you'll want to use a shader for this.  The basis for your technique will be the EYE_LINEAR TexGen mode that old-fashioned projective texturing used, so you'll probably want to read up on that.  There's some sample code written in pure OpenGL here:

http://www.sgi.com/products/software/opengl/examples/glut/advanced/source/projtex.c

The equation used for EYE_LINEAR TexGen is given in the OpenGL spec.  You can also find it in the man page for glTexGen, available here:

http://www.opengl.org/sdk/docs/man2/xhtml/glTexGen.xml


Once you're familiar with that technique, you'll probably be able to come up with a specific technique that works better for your situation.

Another benefit of using shaders is that you'll be able to do any blending, exposure compensation, etc. that you might be needing to do really easily as part of the texturing process.

I wanted to avoid splitting the mesh, at least for the internal representation (which I hoped included visualization). Pros and cons have been discussed in this thread (in case you are interested)

https://groups.google.com/d/topic/reconstructme/sDb_A-n6_A0/discussion

You might not need to segment the mesh.  If you don't segment the mesh, it means that you'll have to have all of your photo textures active at the same time.  Most modern graphics cards can handle at least 8, decent mid range gaming cards can handle as many as 64, and the high-end enthusiast cards can even hit 128.  If you're photo count is less than this number for your hardware, you'll probably be OK.  You'll just need to encode which photo or photos are important for each vertex, so you can look them up in the shader, you'd do this as a vertex attribute.

Your photo texture samplers will be one set of uniforms, and you'll need another set to encode the photo position and orientation, as these will be needed to calculate the texture coordinates.  You won't need to pass texture coordinates as vertex attributes, because you'll be generating them in the vertex shader.  As long as you don't have more than a few photos per vertex, you shouldn't have any issues with the limited number of vertex attributes that can be passed between shader stages.

--"J"

[Christoph Heindl]

On Tue, Oct 23, 2012 at 8:40 PM, Jason Daly <jd...@ist.ucf.edu> wrote:

How can TexGen and a shader help here? Would it allow me to calculate the UV coordinates for a given photo (camera position etc.) and the mesh?

The more I think about it, the more I think you'll want to use a shader for this.  The basis for your technique will be the EYE_LINEAR TexGen mode that old-fashioned projective texturing used, so you'll probably want to read up on that.  There's some sample code written in pure OpenGL here:

http://www.sgi.com/products/software/opengl/examples/glut/advanced/source/projtex.c

The equation used for EYE_LINEAR TexGen is given in the OpenGL spec.  You can also find it in the man page for glTexGen, available here:

http://www.opengl.org/sdk/docs/man2/xhtml/glTexGen.xml

Thanks for the hints. From browsing the documentation it seems though that this would also texture non-visible triangles (i.e backfacing triangles)? This however would lead to messy texturing, since I have photos from around the mesh.

 

Once you're familiar with that technique, you'll probably be able to come up with a specific technique that works better for your situation.

Another benefit of using shaders is that you'll be able to do any blending, exposure compensation, etc. that you might be needing to do really easily as part of the texturing process.

I think I'd need a starter sample for how to use EYE_LINEAR in combination with a shader.

 

You might not need to segment the mesh.  If you don't segment the mesh, it means that you'll have to have all of your photo textures active at the same time.  Most modern graphics cards can handle at least 8, decent mid range gaming cards can handle as many as 64, and the high-end enthusiast cards can even hit 128.  If you're photo count is less than this number for your hardware, you'll probably be OK.  You'll just need to encode which photo or photos are important for each vertex, so you can look them up in the shader, you'd do this as a vertex attribute.

Since ReconstructMe itself requires a decent graphics card we are ok with that approach. I assume that there won't be more than 64 fotos per mesh and I assume that a vertex would never be textured from more than 3 fotos.

 

Your photo texture samplers will be one set of uniforms, and you'll need another set to encode the photo position and orientation, as these will be needed to calculate the texture coordinates.  You won't need to pass texture coordinates as vertex attributes, because you'll be generating them in the vertex shader.  As long as you don't have more than a few photos per vertex, you shouldn't have any issues with the limited number of vertex attributes that can be passed between shader stages.

That sounds interesting. You don't have an example at hands? 

A rather related problem is the export of meshes. Using the shader approach, how would one deliver such a textured mesh? Wouldn't it make more sense to  pre-calucate the UV coordinates for each vertex/foto infront (incl. visibility detection) and pass this to the shader (possible?), so that I could decide on file export (based on the format chosen) to either

 - split the mesh if the format does not support multiple texture layers (.ply/.obj maybe).

 - don't split the mesh if the format supports multiple layers (.fbx)

Thanks for your time,

Christoph

[Jason Daly]

On 10/24/2012 03:03 AM, Christoph Heindl wrote:

Thanks for the hints. From browsing the documentation it seems though that this would also texture non-visible triangles (i.e backfacing triangles)? This however would lead to messy texturing, since I have photos from around the mesh.

Shouldn't be a problem if you don't render back faces (which is the default).  The fragment shader won't even run for those pixels, as they'll already have been culled.

Once you're familiar with that technique, you'll probably be able to come up with a specific technique that works better for your situation.

Another benefit of using shaders is that you'll be able to do any blending, exposure compensation, etc. that you might be needing to do really easily as part of the texturing process.

I think I'd need a starter sample for how to use EYE_LINEAR in combination with a shader.

 

Here's a page from the nVidia Cg tutorial that talks about projective texturing, including the texture coordinate generation.  The example code is in Cg, but it shouldn't be too hard to port to OpenGL (vec4 instead of float4, things like that).  Best I could do in a few minutes.

http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter09.html


If you Google around a bit, I'm sure you can find other examples.

Your photo texture samplers will be one set of uniforms, and you'll need another set to encode the photo position and orientation, as these will be needed to calculate the texture coordinates.  You won't need to pass texture coordinates as vertex attributes, because you'll be generating them in the vertex shader.  As long as you don't have more than a few photos per vertex, you shouldn't have any issues with the limited number of vertex attributes that can be passed between shader stages.

That sounds interesting. You don't have an example at hands?

Afraid not, I've never actually done it myself  :-)

There is probably at least a basic example online somewhere, though.

A rather related problem is the export of meshes. Using the shader approach, how would one deliver such a textured mesh? Wouldn't it make more sense to  pre-calucate the UV coordinates for each vertex/foto infront (incl. visibility detection) and pass this to the shader (possible?), so that I could decide on file export (based on the format chosen) to either

 - split the mesh if the format does not support multiple texture layers (.ply/.obj maybe).

 - don't split the mesh if the format supports multiple layers (.fbx)

Ah, yes.  If you want to end up exporting the mesh, then a real-time shader might be a problem.  It's not impossible, as there are formats like COLLADA that allow you to embed shaders, but from my experience, run-time systems don't often fully support those features.

If you do need to export to a static format (like fbx), then I don't see a way around splitting the mesh.  If your maximum photo count is 64, there's no way you can have all of the textures enabled and statically mapped at the same time (that would mean you'd need 64 sets of texture coordinates, and no graphics card supports that many vertex attributes).

I think you'd need to do an initial pass to see which photos cover which vertices, then divide up the mesh accordingly.  Only one (maybe two or three) textures will need to be active for each patch, which means you'd only need at most three sets of texture coordinates for each vertes.  This should be exportable to one of several modern formats.  Yes, you'll have some duplication of vertices along the boundaries, but this isn't really a huge problem.  The vertex processing most likely isn't going to be your bottleneck here.

Maybe someone smarter than me can come up with a solution that doesn't require splitting the mesh, but I don't see one...

--"J"

[Robert Osfield]

Hi Chirstoph,

I'm surprised by you not getting any replies, guess everyone has been
busy, or perhaps confused by what you are after.

It sounds like to me you just need to use the OSG's standard
multi-texturing support, the OSG will manage handling of OpenGL
versions/extensions for you so you won't need to worry about the
OpenGL backend. All modern OpenGL hardware supports multi-texturing
with the fixed function pipeline and with GLSL shaders, with GLSL you
have greater flexibility and range of units that you can use.

How many textures do you want to assign to a single mesh?

W.r.t projective texturing examples have a look the OSG's osg::TexGen
(which wraps up glTexGen), osg::TexGenNode and the examples that use
TexGen.

~/OpenSceneGraph/examples$ grep TexGen */* -l
osgcubemap/osgcubemap.cpp
osgmultitexture/osgmultitexture.cpp
osgoit/osgoit.cpp
osgprerendercubemap/osgprerendercubemap.cpp
osgspotlight/osgspotlight.cpp
osgteapot/osgteapot.cpp
osgtexture1D/osgtexture1D.cpp
osgtexture3D/osgtexture3D.cpp
osgtexturecompression/osgtexturecompression.cpp
osgvertexprogram/osgvertexprogram.cpp
osgvirtualprogram/CreateAdvancedHierachy.cpp
osgvirtualprogram/CreateSimpleHierachy.cpp
osgvirtualprogram/osgvirtualprogram.cpp
osgvolume/osgvolume.cpp

Robert.

> _______________________________________________
> osg-users mailing list
> osg-...@lists.openscenegraph.org
> http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org
>
_______________________________________________
osg-users mailing list
osg-...@lists.openscenegraph.org
http://lists.openscenegraph.org/listinfo.cgi/osg-users-openscenegraph.org

[Christoph Heindl]

Hi Robert,

On Thu, Oct 25, 2012 at 11:11 AM, Robert Osfield <robert....@gmail.com> wrote:

Hi Chirstoph,

I'm surprised by you not getting any replies, guess everyone has been
busy, or perhaps confused by what you are after.

thanks for your reply. Actually my post seems to show up twice on this list under the same topic. I think it was my fault, as I did a resend when the first did not seem to show up.

 

It sounds like to me you just need to use the OSG's standard
multi-texturing support, the OSG will manage handling of OpenGL
versions/extensions for you so you won't need to worry about the
OpenGL backend.  All modern OpenGL hardware supports multi-texturing
with the fixed function pipeline and with GLSL shaders, with GLSL you
have greater flexibility and range of units that you can use.

Ok, so the fixed function pipeline does already contain an implementation for multitexturing, in contrast to the programmable pipeline via shaders, right?

 

How many textures do you want to assign to a single mesh?

I would assume that ten fotos should suffice to texture a mesh 95%. I need to deal with vertices that are textured from multiple fotos or are never textured somehow on my side. Is there a way (without splitting the mesh) to assign a default material (simple color) to all untextured vertices/triangles? I'd assume that I can assign out of range UV coordinates (-1) and set a border color.

 

Thanks for your attention.

Best,

Christoph

[Christoph Heindl]

On Wed, Oct 24, 2012 at 5:30 PM, Jason Daly <jd...@ist.ucf.edu> wrote:

On 10/24/2012 03:03 AM, Christoph Heindl wrote: 

Ah, yes.  If you want to end up exporting the mesh, then a real-time shader might be a problem.  It's not impossible, as there are formats like COLLADA that allow you to embed shaders, but from my experience, run-time systems don't often fully support those features.

Ok.

 

If you do need to export to a static format (like fbx), then I don't see a way around splitting the mesh.  If your maximum photo count is 64, there's no way you can have all of the textures enabled and statically mapped at the same time (that would mean you'd need 64 sets of texture coordinates, and no graphics card supports that many vertex attributes).

I think you'd need to do an initial pass to see which photos cover which vertices, then divide up the mesh accordingly.  Only one (maybe two or three) textures will need to be active for each patch, which means you'd only need at most three sets of texture coordinates for each vertes.  This should be exportable to one of several modern formats.  Yes, you'll have some duplication of vertices along the boundaries, but this isn't really a huge problem.  The vertex processing most likely isn't going to be your bottleneck here.

I agree, but I think for further mesh processing duplicate vertices could be problem (decimation, ...).

 

Maybe someone smarter than me can come up with a solution that doesn't require splitting the mesh, but I don't see one...

Just one more thought. In my research I found that most modelling software support UV unwrapping (not sure if it is the right term). What I mean is that the user marks a seam on the mesh and the mesh is then unfolded along that seem into individual patches that can be mapped to the UV space. Do you know how such seams are handled by the rendering engine? It seems naturally to me that at seems one need to split the mesh and duplicate vertices.

Best & thanks for your help,

Christoph

[Robert Osfield]

HI Chirstoph,

On 25 October 2012 17:13, Christoph Heindl <christop...@gmail.com> wrote:
> Ok, so the fixed function pipeline does already contain an implementation
> for multitexturing, in contrast to the programmable pipeline via shaders,
> right?

Both fixed function and shaders support multi-texturing. The fixed
function is typically more limited in the number of texture units the
driver exposes, how many units each support is driver and hardware
dependent.

>> How many textures do you want to assign to a single mesh?
>
> I would assume that ten fotos should suffice to texture a mesh 95%.

This might well push you over the limit supported by fixed function
pipeline on your hardware so it's likely you'll need to look at using
GLSL shaders. As you have a pretty straight forward usage case this
won't require anything too complicated.

> I need
> to deal with vertices that are textured from multiple fotos or are never
> textured somehow on my side. Is there a way (without splitting the mesh) to
> assign a default material (simple color) to all untextured
> vertices/triangles? I'd assume that I can assign out of range UV coordinates
> (-1) and set a border color.

With OpenGL the whole mesh will be textured by all textures you apply,
the way to get non and textured parts for areas outside the texture
coordinate range 0,0 to 1,1 is to set a alpha value of 0 for the
border colour, something like (1,1,1,0) would be appropriate. However
if you are using GLSL you can use test the texture coordinate prior to
reading the texture and then just blend the colours as you require.

It's not really me place to teach you GLSL so I have to recommend that
you look at online resources for GLSL, and on the OSG side have a look
at the osgshaders example and any other places where osg::Program
crops up in the OSG examples or code base.

Robert.

[Christoph Heindl]

Hi Robert,

On Thu, Oct 25, 2012 at 6:26 PM, Robert Osfield 

This might well push you over the limit supported by fixed function
pipeline on your hardware so it's likely you'll need to look at using
GLSL shaders.  As you have a pretty straight forward usage case this
won't require anything too complicated.

Ok thanks

 

With OpenGL the whole mesh will be textured by all textures you apply,
the way to get non and textured parts for areas outside the texture
coordinate range 0,0 to 1,1 is to set a alpha value of 0 for the
border colour, something like (1,1,1,0) would be appropriate.  However
if you are using GLSL you can use test the texture coordinate prior to
reading the texture and then just blend the colours as you require.

This seems the way to go. Alternatively one could split (i.e duplicate vertices) the mesh at seams and generate a texture atlas (should be easy for all the fotos). This should allow me to use one texture.

Multiple approaches, which one to take? ;)

 

It's not really me place to teach you GLSL so I have to recommend that
you look at online resources for GLSL, and on the OSG side have a look
at the osgshaders example and any other places where osg::Program
crops up in the OSG examples or code base.

That's ok, I'm thankful for all the input received! 

Best,

Christoph

[Jason Daly]

On 10/25/2012 12:17 PM, Christoph Heindl wrote:

I think you'd need to do an initial pass to see which photos cover which vertices, then divide up the mesh accordingly.  Only one (maybe two or three) textures will need to be active for each patch, which means you'd only need at most three sets of texture coordinates for each vertes.  This should be exportable to one of several modern formats.  Yes, you'll have some duplication of vertices along the boundaries, but this isn't really a huge problem.  The vertex processing most likely isn't going to be your bottleneck here.

I agree, but I think for further mesh processing duplicate vertices could be problem (decimation, ...).

That's possible.  If you're relying on a single mesh for your decimation process, then it could very well present difficulties.  The decision will probably have to be based on what combination of techniques presents the fewest difficulties  :-)

 

Maybe someone smarter than me can come up with a solution that doesn't require splitting the mesh, but I don't see one...

Just one more thought. In my research I found that most modelling software support UV unwrapping (not sure if it is the right term). What I mean is that the user marks a seam on the mesh and the mesh is then unfolded along that seem into individual patches that can be mapped to the UV space. Do you know how such seams are handled by the rendering engine? It seems naturally to me that at seems one need to split the mesh and duplicate vertices.

I'm not a modeler, but I think I've seen the tool you're referring to.  It's typically used to allow a digital artist to design a texture atlas for the specific mesh.  The tool creates a "blank" texture atlas with all of the texture coordinates mapped appropriately, and then the artist simply paints the desired imagery onto that "blank" atlas.  In your case, you've already got imagery that you're trying to map onto a mesh, so you'd have to warp your photos to fit the texture atlas.  Plus, you indicated you'd be needing to blend between textures at the photo borders.  With this in mind, I don't know if that tool will be more likely to help or hinder you, to be honest.

--"J"

[Glenn Waldron]

I would try multipass rendering first. It is likely to be the slowest, but also probably the easiest to implement and you don't have to worry about exceeding your hardware limits. This falls into the category of "just get it working" and then worry about optimizing it later if necessary. Who knows -- maybe the performance will be acceptable.

Glenn Waldron / @glennwaldron

[Christoph Heindl]

On Thu, Oct 25, 2012 at 9:32 PM, Glenn Waldron <gwal...@gmail.com> wrote:

I would try multipass rendering first. It is likely to be the slowest, but also probably the easiest to implement and you don't have to worry about exceeding your hardware limits. This falls into the category of "just get it working" and then worry about optimizing it later if necessary. Who knows -- maybe the performance will be acceptable.

How would I go about this? I assume I'd need to enable different textures on each pass. How the pass the UV coordinates (as different sets)?

Best,

Christoph

[Glenn Waldron]

Right. So for multipass in general, you have a root node; then under that node there is one Group node per pass; then all these Group nodes share a common child (the geometry to render). Each per-pass Group node can hold a unique StateSet that assigns the proper texture, etc.

Since you will also need to use the TexGen capabilities (discussed earlier in this thread) to generate the correct texture coordinates for each pass, you might use an osg::TexGenNode in place of the osg::Group. TexGen will generate the UV coordinates for you.

You might also look at the osgFX::Effect class. This is a framework for rendering a subgraph multiple times (multipass), using a separate StateSet for each pass (just what I described above) so it might be a good fit, or at least good reference material.

HTH.

Glenn Waldron / Pelican Mapping / @glennwaldron

[Christoph Heindl]

On Fri, Oct 26, 2012 at 3:57 PM, Glenn Waldron <gwal...@gmail.com> wrote:

Right. So for multipass in general, you have a root node; then under that node there is one Group node per pass; then all these Group nodes share a common child (the geometry to render). Each per-pass Group node can hold a unique StateSet that assigns the proper texture, etc.

Very clever. Thanks for sharing

 

Since you will also need to use the TexGen capabilities (discussed earlier in this thread) to generate the correct texture coordinates for each pass, you might use an osg::TexGenNode in place of the osg::Group. TexGen will generate the UV coordinates for you.

I don't think so, as I already have the UV coordinates. I calculate them myself in order to allow for camera lens distortions and such. I think that TexGen (EYE_LINEAR) mentioned previously assumes a plain pinhole camera model without regarding effects from lens distortion.

You might also look at the osgFX::Effect class. This is a framework for rendering a subgraph multiple times (multipass), using a separate StateSet for each pass (just what I described above) so it might be a good fit, or at least good reference material.

Sounds good, just the name (Effect) puzzles me. From looking at the documentation it seems that actually a lot of effects such as outlines, glow etc are done with that.

Best,

Christoph

[Robert Osfield]

Hi Christoph,

On 26 October 2012 17:04, Christoph Heindl <christop...@gmail.com> wrote:
>> Since you will also need to use the TexGen capabilities (discussed earlier
>> in this thread) to generate the correct texture coordinates for each pass,
>> you might use an osg::TexGenNode in place of the osg::Group. TexGen will
>> generate the UV coordinates for you.
>
>
> I don't think so, as I already have the UV coordinates. I calculate them
> myself in order to allow for camera lens distortions and such. I think that
> TexGen (EYE_LINEAR) mentioned previously assumes a plain pinhole camera
> model without regarding effects from lens distortion.

If you use your own shaders then you'll be able to compute the ST
coordinates (what OpenGL calls the UV coords) and do something more
sophisticated than standard OpenGL TexGen.

For cases where the ST coordinates can't be computed in object or eye
coordinates then you'll need to rely upon the UV coords that you've
computed. The OSG supports as many texture units as the underlying
OpenGL implementation supports and will scale up to 8 without problem
on most hardware/drivers. Going this route would be the most straight
forward.

> Sounds good, just the name (Effect) puzzles me. From looking at the
> documentation it seems that actually a lot of effects such as outlines, glow
> etc are done with that.

I wouldn't recommend considering the osgFX library, it'd just
overcomplicated things for no gain.

Also while you could go the multi-pass route I would recommend it as
the multi-texturing route should get you far enough along without
needing to go for a more complicated multi-pass solution. The OSG
supports multi-pass cleanly but it's always more complicated than
using multi-texturing.

Robert.

[Glenn Waldron]

On Fri, Oct 26, 2012 at 12:04 PM, Christoph Heindl <christop...@gmail.com> wrote:

On Fri, Oct 26, 2012 at 3:57 PM, Glenn Waldron <gwal...@gmail.com> wrote:

Right. So for multipass in general, you have a root node; then under that node there is one Group node per pass; then all these Group nodes share a common child (the geometry to render). Each per-pass Group node can hold a unique StateSet that assigns the proper texture, etc.

Very clever. Thanks for sharing

 

Since you will also need to use the TexGen capabilities (discussed earlier in this thread) to generate the correct texture coordinates for each pass, you might use an osg::TexGenNode in place of the osg::Group. TexGen will generate the UV coordinates for you.

I don't think so, as I already have the UV coordinates. I calculate them myself in order to allow for camera lens distortions and such. I think that TexGen (EYE_LINEAR) mentioned previously assumes a plain pinhole camera model without regarding effects from lens distortion.

How are they stored? Per vertex? Or is it some kind of regular grid that does distortion correction? Is the computation something you could do in a vertex shader if the shader can access to the camera parameters?

 

You might also look at the osgFX::Effect class. This is a framework for rendering a subgraph multiple times (multipass), using a separate StateSet for each pass (just what I described above) so it might be a good fit, or at least good reference material.

Sounds good, just the name (Effect) puzzles me. From looking at the documentation it seems that actually a lot of effects such as outlines, glow etc are done with that.

True; I just meant that it uses the same StateSet-per-pass idea.

Glenn

[Glenn Waldron]

On Fri, Oct 26, 2012 at 12:34 PM, Robert Osfield <robert....@gmail.com> wrote:

Hi Christoph,

On 26 October 2012 17:04, Christoph Heindl <christop...@gmail.com> wrote:
>> Since you will also need to use the TexGen capabilities (discussed earlier
>> in this thread) to generate the correct texture coordinates for each pass,
>> you might use an osg::TexGenNode in place of the osg::Group. TexGen will
>> generate the UV coordinates for you.
>
>
> I don't think so, as I already have the UV coordinates. I calculate them
> myself in order to allow for camera lens distortions and such. I think that
> TexGen (EYE_LINEAR) mentioned previously assumes a plain pinhole camera
> model without regarding effects from lens distortion.

If you use your own shaders then you'll be able to compute the ST
coordinates (what OpenGL calls the UV coords) and do something more
sophisticated than standard OpenGL TexGen.

For cases where the ST coordinates can't be computed in object or eye
coordinates then you'll need to rely upon the UV coords that you've
computed.  The OSG supports as many texture units as the underlying
OpenGL implementation supports and will scale up to 8 without problem
on most hardware/drivers.  Going this route would be the most straight
forward.

Another idea is to use a TextureArray (GL_EXT_texture_array). This removes the limit on the number of textures, but adds the constraint that they all must be the same size and that you need GL 2.0.

[Jason Daly]

On 10/26/2012 01:19 PM, Glenn Waldron wrote:

For cases where the ST coordinates can't be computed in object or eye
coordinates then you'll need to rely upon the UV coords that you've
computed.  The OSG supports as many texture units as the underlying
OpenGL implementation supports and will scale up to 8 without problem
on most hardware/drivers.  Going this route would be the most straight
forward.

Another idea is to use a TextureArray (GL_EXT_texture_array). This removes the limit on the number of textures, but adds the constraint that they all must be the same size and that you need GL 2.0.

From what I can tell, the main issue isn't the number of textures, it's the number of varying parameters between the vertex and fragment shader.  If there are too many textures being applied at once (each with their own set of texture coordinates), it may be possible to run out of these, depending on what else is being interpolated between shader stages.

On my machine (Geforce GTX 260):

GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS=96
GL_MAX_VARYING_FLOATS=60

So, 96 textures, but only 60 floats for varying parameters.  This seems like a lot, but if there are 64 textures on the mesh, it's not even close to enough.  Of course, newer cards than mine probably have more...

--"J"

[Christoph Heindl]

On Fri, Oct 26, 2012 at 7:19 PM, Glenn Waldron <gwal...@gmail.com> wrote:

Another idea is to use a TextureArray (GL_EXT_texture_array). This removes the limit on the number of textures, but adds the constraint that they all must be the same size and that you need GL 2.0.

What size are you referring to (size of foto/textor or size of vertex uv coordiante array)?

 

Best,

Christoph

[Christoph Heindl]

On Sat, Oct 27, 2012 at 2:13 AM, Jason Daly <jd...@ist.ucf.edu> wrote:

On 10/26/2012 01:19 PM, Glenn Waldron wrote:

 

From what I can tell, the main issue isn't the number of textures, it's the number of varying parameters between the vertex and fragment shader.  If there are too many textures being applied at once (each with their own set of texture coordinates), it may be possible to run out of these, depending on what else is being interpolated between shader stages.

I could create a single texture from all fotos (by stitching them together if that helps). Then, however, it is unclear to me how to handle shading at using one texture and multiple UV/ST arrays.

[Christoph Heindl]

On Fri, Oct 26, 2012 at 7:11 PM, Glenn Waldron <gwal...@gmail.com> wrote:

How are they stored? Per vertex? Or is it some kind of regular grid that does distortion correction? Is the computation something you could do in a vertex shader if the shader can access to the camera parameters?

Per vertex. UV coordinates for vertices that are not textured from a given foto are set to a special value (-1).

The computation could be done in a shader I suppose, though I'm not completely sure about the constraints of a shader. I'm used to general purpose computing on GPUs in cuda and opencl and in both languages its possible: it requires at least a loop construct and floating point arithmetic.

On the CPU I additionally perform a visibility test for the vertex, but as others have noted invisible vertices wouldn't get in into the vertex shader stage.

This sounds like a clever idea, but I have to keep exporting the mesh in mind. When exporting I think I should have the UV coordinates ready on the CPU. Is there a way to transmit the output of the shader (i.e vertex ST coordinates) back to the CPU?

Best,

Christoph

[Glenn Waldron]

The texture size. -gw

[Jason Daly]

On 10/27/2012 02:09 AM, Christoph Heindl wrote:
>
> This sounds like a clever idea, but I have to keep exporting the mesh
> in mind. When exporting I think I should have the UV coordinates ready
> on the CPU. Is there a way to transmit the output of the shader (i.e
> vertex ST coordinates) back to the CPU?

You can do this with transform feedback. I'm not sure how good OSG's
support for this feature is, though.

--"J"