Vertex texture support

[alanmechy]

Hi

Now that both FF and Chrome will be using Angle by default, is there
any chance that support for vertex textures could be added? What would
be involved in doing that? The OGL ES 2 default minimum for
MAX_VERTEX_TEXTURE_IMAGE_UNITS is, of course, 0 and that's what Angle
currently seems to return.

Because vertex textures are specified as part of OGL 2 but the minimum
is 0, I don't think this is an extension, just something that Angle
doesn't implement. I'm not an expert on DX9, so I have no idea what
would be involved in adding the capability. I have graphics cards
which provide support for VTUs and seem to work OK with WebGL.

I've seen a few posts on the web asking about this, and there are some
really cool things that you can do with vertex textures (eg certain
kinds of animation data can be loaded via a texture.)

In some cases the underlying hardware might not support it - in which
case Angle could return 0.

I might be prepared to 'have a go' myself, maybe someone could outline
what would be required. If its at all possible to implement, let me
know, and I'll post an enhancement request.

Regards

Alan

[Daniel Koch]

Hi Alan,

We did look into this a few months back, and it is possible to do so, but there are some limitations in D3D9 that make this rather annoying to implement for ANGLE.

The D3D9 API only exposes support for 4 vertex texture units, so that is the most that could be advertised for MAX_VERTEX_TEXTURE_IMAGE_UNITS. 

The biggest issue though, is that the actual hardware support for VTF is rather limited.  OpenGL doesn't provide any restrictions on what formats and/or types of textures can used for vertex textures, or if filtering can be enabled, whereas D3D9 does.  Exactly what is supported is hardware dependent.  In the worst case when VTF _is_ supported on D3D9, only 2D 32-bit floating-point textures are required and only with point filtering.   

This leaves an open question of how to handle VTF for non-FP32 texture formats, CUBE (and eventually 3D) texture types, and filtering.  The most likely solution here is likely to fall back to software vertex processing in the unsupported cases.  This is likely what the OpenGL drivers are doing on these cards (and appears to the source of much frustration according to some posts on the WebGL mailing list).  This does add in the extra complexity of potentially needing to move or copy texture data between texture memory pools. 

If this is something you (or someone else) is interested in tackling, I can provide more details about the various pitfalls, options and exactly what needs to be done.

Hope this helps,

Daniel

---

                        Daniel Koch -+- dan...@transgaming.com

Senior Graphics Architect -+- TransGaming Inc.  -+- www.transgaming.com

[Aras Pranckevicius]

The biggest issue though, is that the actual hardware support for VTF is rather limited.  OpenGL doesn't provide any restrictions on what formats and/or types of textures can used for vertex textures, or if filtering can be enabled, whereas D3D9 does.  Exactly what is supported is hardware dependent.  In the worst case when VTF _is_ supported on D3D9, only 2D 32-bit floating-point textures are required and only with point filtering.   

Yes, according to the spec & D3D9 caps most GPUs only support some set of floating point formats for vertex texturing.

However, in reality I found them to be much more flexible on D3D9. Basically, all DX10+ level hardware does support sampling textures in the vertex shader for pretty much all formats (it has to, otherwise it wouldn't be DX10 compliant). On all DX10+ GPUs from NVIDIA, AMD and Intel I tried (and I tried a lot), they seem to be able to do that on D3D9 as well. Even if for some reason no one updated the caps reporting (the caps still say it can't...).

For DX10+ levels GPUs it goes quite far actually: even FOURCC hack texture formats (like native depth buffer aka INTZ) can be sampled in the vertex shader on D3D9.

On DX9 SM3.0 level hardware the support is quite limited though. None of AMD or Intel cards support it (caps reports zero vertex samplers), and NVIDIA cards really only support a couple of floating point formats. What we do at Unity, is just treat those as "does not support vertex textures".

--
Aras Pranckevičius
work: http://unity3d.com
home: http://aras-p.info

[al...@mechnicality.com]

Hi Aras

Very useful information! I've made one comment inline.

So this means that effectively it will only work on Vista and Windows 7? AFAIR XP doesn't support anything later than DX9. And, Daniel,  how does Angle handle DX10+. I assume that at the moment you only query for the DX9 COM ifc? Its awhile since I looked at the code.

Just to be clear, Aras' comment seems to imply that what one would have to do is to query for DX10, determine whether or not there was VTF support, and then actually use that as the capability.

Regards

Alan

--
Aras Pranckevičius
work: http://unity3d.com
home: http://aras-p.info

-- 
Alan Chaney
CTO and Founder, Mechnicality, Inc.
www.mechnicality.com

[Aras Pranckevicius]

So this means that effectively it will only work on Vista and Windows 7? AFAIR XP doesn't support anything later than DX9. And, Daniel,  how does Angle handle DX10+. I assume that at the moment you only query for the DX9 COM ifc? Its awhile since I looked at the code.

No, what I said is that all DX10 capable GPUs do seem to support vertex texture sampling from all texture formats. You're not required to use DX10, this works under DX9 as well.

So the only remaining problem is, how to detect if a GPU is DX10 capable. On Vista or later that's quite easy; you could try instantiating a D3D10 device and check if that succeeds.

Alternatively, you could use some heuristics. For Unity, we just detect vertex texture capability of R16F format, this seems to be present for all DX10+ GPUs, and absent for older GPUs that only support a limited set of vertex texture formats. Paste from our code:

// In theory, vertex texturing is texture format dependent. However, in practice the caps lie,

// especially on NVIDIA hardware.

//

// ATI cards: all DX10+ GPUs report all texture formats as vertex texture capable (good!)

// Intel cards: all SM3.0+ GPUs report all texture formats as vertex texture capable (good!)

// NV cards: all DX10+ GPUs report only floating point formats as capable, but all others actually work as well.

//           GeForce 6&7 only report R32F and A32R32G32B32F, and only those work.

//

// So we check for R16F support; this will return true on all GPUs that can handle ALL

// texture formats.

hasVertexTextures =

    ((LOWORD(d3d.d3dcaps.VertexShaderVersion) >= (3<<8)+0)) &&

    SUCCEEDED(d3dobject->CheckDeviceFormat(

        d3dAdapter, d3dDevType, GetD3DFormatForChecks(),

        D3DUSAGE_QUERY_VERTEXTEXTURE, D3DRTYPE_TEXTURE, D3DFMT_R16F));

[al...@mechnicality.com]

On 12/8/2010 11:50 AM, Aras Pranckevicius wrote:

So this means that effectively it will only work on Vista and Windows 7? AFAIR XP doesn't support anything later than DX9. And, Daniel,  how does Angle handle DX10+. I assume that at the moment you only query for the DX9 COM ifc? Its awhile since I looked at the code.

No, what I said is that all DX10 capable GPUs do seem to support vertex texture sampling from all texture formats. You're not required to use DX10, this works under DX9 as well.

Ok, thanks for clarifying and for the tip - I await further comments from Daniel.

Regards

Alan

[Steve]

Hi guys,

Speaking as a user, I have a couple upcoming projects that would be
greatly helped by vertex textures. Having you look into this feature
is greatly appreciated!

Do you think that it would be possible to just determine if regular,
2D, float textures are supported at all (regardless of # of units)?
Some method that would return 'hasVertexTextures' from Aras's message?
I don't believe that any NV card that supports it has fewer than 4
units. Maybe just provide an artificially low cap of 4 until a better
or simpler approach can be found.

Having a vertex shader throw a compiler error on texture2D (which is
currently the case) is frustrating, as it's a very useful feature that
a lot of hardware can handle. Although I'm probably not the guy to
modify the C code, please let me know if I can do anything to help.

Again, thanks for looking into it!

-Steve


On Dec 8, 1:56 pm, "a...@mechnicality.com" <a...@mechnicality.com>
wrote:

> On 12/8/2010 11:50 AM, Aras Pranckevicius wrote:
>
> >     So this means that effectively it will only work on Vista and Windows 7? AFAIR XP doesn't
> >     support anything later than DX9. And, Daniel,  how does Angle handle DX10+. I assume that at
> >     the moment you only query for the DX9 COM ifc? Its awhile since I looked at the code.
>

> > No, what I said is that all DX10 /capable/ GPUs do seem to support vertex texture sampling from

[Daniel Koch]

Hi Aras,

Thanks for the information.  I didn't realize that the DX10+ NVIDIA cards supported it even though they didn't report the caps support.

This will definitely simplify the implementation if we don't have to fall back to software vertex processing (and it is likely more desirable all around!).

Daniel

[Daniel Koch]

On 2010-12-11, at 12:09 AM, Steve wrote:

Hi guys,

Speaking as a user, I have a couple upcoming projects that would be
greatly helped by vertex textures. Having you look into this feature
is greatly appreciated!

Do you think that it would be possible to just determine if regular,
2D, float textures are supported at all (regardless of # of units)?

It's certainly possible to detect this yes.

Some method that would return 'hasVertexTextures' from Aras's message?

We could always add an ANGLE extension which provides a new glGet* command to report this.

However, do you really think there is enough benefit to enable only this subset of the functionality specifically for Geforce 6 and 7 series cards?

Keep in mind the WebGL 1.0 (assuming you're talking about WebGL uses of ANGLE here) doesn't actually expose support for floating point textures.

I don't believe that any NV card that supports it has fewer than 4
units. Maybe just provide an artificially low cap of 4 until a better
or simpler approach can be found.

In D3D9 you either have 0 or 4 units.  There are no more than that exposed in the API and so > 4 will not be possible (without writing a D3D10 backend!)

Having a vertex shader throw a compiler error on texture2D (which is
currently the case) is frustrating, as it's a very useful feature that

We could support the functions, but then the shader would likely just fail silently.  Would that actually be an improvement?

Either way, you need to query for MAX_VERTEX_TEXTURE_IMAGE_UNITS to write portable code.  And yes -- you really should try to write portable code!

[Steve]

The topic was discussed on this Chromium thread -
http://code.google.com/p/chromium/issues/detail?id=65340

Another issue was brought up. If, as you discussed,
MAX_VERTEX_TEXTURE_IMAGE_UNITS is accurately reported, the error for
when it's exceeded could be improved. Currently it looks like a syntax
error (which is confusing) - 'texture2D': no matching overloaded
function found

Here's a quote from the thread:
---------------
I think it is not reporting the correct compilation error. The error
is not that the texture2D intrinsic function is not available. The
error is that the vertex shader has exceeded the number of available
samplers (which is zero). If this is correct, for consistency with
fragment shaders, the error would be something along the lines of:

"error: maximum number of samplers exceeded."
---------------

-Steve

[Daniel Koch]

Hi folks,

Vertex texture support has now been implemented in ANGLE (as of r645).  This functionality is available in Chrome 13.0.766.0.

I believe the specific error mentioned below should be resolved as well.

Take care,

Daniel