Sampling Texture in Vertex Shader using tex2Dlod
Wyck
rendering code, but it's working in other pieces of code. Also, the effect's
sampler is returning correct values in the pixel shader, but not in the
vertex shader, so it's not that my texture data is wrong either.
I'm trying to identify what the differences are, and it seems to be the
texture formats.
Are there different texture formats that are unusable when doing a tex
lookup in the vertex shader compared to what can be sampled in the pixel
shader?
I looked in the caps at the D3DUSAGE_QUERY_VERTEXTEXTURE and it says that
only the float formats are supported, but that seems impossible to be true
because I can sample D3DFMT_X8R8G8B8 textures from the vertex shader in other
code. What's am I missing?
Again, the baffling part is that the pixel shader using the SAME sampler as
the vertex shader but the tex2Dlod returns (0,0,0,0) in the vertex shader,
and correct vals in the pixel shader.
- Wyck
Richard [Microsoft Direct3D MVP]
=?Utf-8?B?V3ljaw==?= <Wy...@discussions.microsoft.com> spake the secret code
<73252315-0CC1-4DAF...@microsoft.com> thusly:
Standard questions apply:
- debug runtime installed?
- debug runtime selected?
- all HRESULTs checked?
- no relevant messages in the debug output stream?
- works on REF?
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
<http://www.xmission.com/~legalize/book/download/index.html>
Pilgrimage 2006, Saturday September 23rd: <http://pilgrimage.scene.org>
North America's Longest Running Demoparty
Wyck
> [Please do not mail me a copy of your followup]
>
> =?Utf-8?B?V3ljaw==?= <Wy...@discussions.microsoft.com> spake the secret code
> <73252315-0CC1-4DAF...@microsoft.com> thusly:
>
> >tex2Dlod is returning (0,0,0,0) for me all the time for certain pieces of
> >rendering code, but it's working in other pieces of code. Also, the effect's
> >sampler is returning correct values in the pixel shader, but not in the
> >vertex shader, so it's not that my texture data is wrong either.
> >
> >I'm trying to identify what the differences are, and it seems to be the
> >texture formats.
> >
> >Are there different texture formats that are unusable when doing a tex
> >lookup in the vertex shader compared to what can be sampled in the pixel
> >shader?
> >
> >I looked in the caps at the D3DUSAGE_QUERY_VERTEXTEXTURE and it says that
> >only the float formats are supported, but that seems impossible to be true
> >because I can sample D3DFMT_X8R8G8B8 textures from the vertex shader in other
> >code. What's am I missing?
> >
> >Again, the baffling part is that the pixel shader using the SAME sampler as
> >the vertex shader but the tex2Dlod returns (0,0,0,0) in the vertex shader,
> >and correct vals in the pixel shader.
>
> Standard questions apply:
> - debug runtime installed?
Yes.
> - debug runtime selected?
Okay, I've tried both ways.
> - all HRESULTs checked?
Yes, for sure.
> - no relevant messages in the debug output stream?
Ok, in ref it says you can't lock a texture unless its dynamic, but I'm not
locking textures.
> - works on REF?
After I marked my texture as D3DUSAGE_DYNAMIC, it started working correctly
in REF. But still doesn't work in HAL.
Card is nVidia 7900 GT.
Richard [Microsoft Direct3D MVP]
Can you post the assembly code for your shader? The command-line
compiler tool has an option to output an assembly listing for your
HLSL.
Wyck
> [Please do not mail me a copy of your followup]
>
> Can you post the assembly code for your shader? The command-line
> compiler tool has an option to output an assembly listing for your
> HLSL.
//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.12.589.0000
//
// fxc test.hlsl /EVShade /Tvs_3_0
//
//
// Parameters:
//
// sampler2D samplerB;
//
//
// Registers:
//
// Name Reg Size
// ------------ ----- ----
// samplerB s0 1
//
vs_3_0
def c0, 1, 0, 0, 0
dcl_position v0
dcl_2d s0
dcl_position o0
dcl_texcoord o1.xy
mul r0, c0.xxyy, v0.xyxx
texldl r0, r0, s0
mov o0.xy, r0
mov o0.zw, v0
mov o1.xy, v0
// approximately 6 instruction slots used (2 texture, 4 arithmetic)
The gist of this test shader above is that it samples a texture, and uses
the xy value of the colour as position coordinates. Works if I use
D3DFMT_A32B32G32R32F texture, but not if I use D3DFMT_A8R8G8B8.
But on the other hand, I am able to sample D3DFMT_A8R8G8B8 textures with the
vertex shader in other situations, for example, this shader (below) doesn't
have a problem reading from D3DFMT_A8R8G8B8 textures, albeit, it's from a
different application, so I'm sure something else is different...that's why
I'm posting, to try to get an idea what else could possibly be different.
//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.12.589.0000
//
// fxc flop.hlsl /EFlop /Tvs_3_0
//
//
// Parameters:
//
// float4x4 mWorldViewProj;
// sampler2D sample;
//
//
// Registers:
//
// Name Reg Size
// -------------- ----- ----
// mWorldViewProj c0 4
// sample s0 1
//
vs_3_0
def c4, 1, 0, 0.333000004, 0
dcl_position v0
dcl_texcoord v1
dcl_2d s0
dcl_position o0
dcl_texcoord o1.xy
mul r0, c4.xxyy, v1.xyxx
texldl r0, r0, s0
mul r0.z, r0.y, c4.z
mov r0.xyw, v0
dp4 o0.x, r0, c0
dp4 o0.y, r0, c1
dp4 o0.z, r0, c2
dp4 o0.w, r0, c3
mov o1.xy, v1
// approximately 10 instruction slots used (2 texture, 8 arithmetic)