Stall on useProgram with the new KHR_parallel_shader_compile

[Jasper St. Pierre]

Hi. I originally asked on the WebGL mailing list at Khronos and was directed here. The original thread, for context, is located here [0]

I am trying to use KHR_parallel_shader_compile in my WebGL app, and am running into a program stall when calling useProgram(). According to engineers on the thread there, the only thing that should cause this is dynamic recompilation, but I don't believe I'm doing anything that would cause that. If I am, however, please let me know what hazards I should avoid. And for the future, having a log message or debug callback that would tell me about bad performance hazards would be appreciated.

If you would like to help test this yourself, please use an extremely recent Chrome Canary, and make sure the flag at chrome://flags/#enable-webgl-draft-extensions is enabled. Reproduction instructions are as follows:

 

1. Navigate to https://noclip.website/#mkwii/beginner_course
2. Wait for the full scene to load.
3. On the left, the button under the file picker should be labeled “Render Hacks”. Open that panel and toggle Enable Vertex Colors or Enable Textures. This causes all the shaders in the scene to recompile.
4. There should be a bit of CPU time spent that, when analyzed in the Profiler, appears to be caused by useProgram.

[0] https://www.khronos.org/webgl/public-mailing-list/public_webgl/1905/msg00010.php

[Geoff Lang]

We do dynamically recompile some shaders in our D3D backend.  This would happen when the program is first used though, probably a draw call.

For example, if you change the data type of your vertex data for the same shader, we have to recompile so the shader can convert the input data to the type it needs (see: VertexExecutable). We do the same if you change which draw buffers are enabled but this is less common.  We try to guess a good initial input signature to compile for but we can't know for sure what state will be used when the program is finally used for a draw.

Do your vertex types change from float/uint/int when the 'Enable Vertex Colors' flag is changed?

--
You received this message because you are subscribed to the Google Groups "angleproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angleproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/3c0decec-b26b-4bdc-b4b9-935b88d8d6c2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jasper St. Pierre]

Hi Geoff,

What do you mean by "change"? I generate and use a number of shaders which use differing amounts of vertex inputs. However, the types of any given input location will remain the same. e.g. layout(location=9) is always a vec3, and layout(location=0), and I always output the full list of slots in each shader. What changes is whether these inputs are used. Unused slots are left unbound on the app side.

When I toggle Vertex Colors, I recompile all shaders. The two variants of each shader use the same inputs, but there's no obvious way to associate one program to the other. However, I do know the vertex inputs and their types at program compilation time. Is there any way to influence the heuristic? Would binding a VAO with the correct inputs at compileShader  / linkProgram time work?

--

  Jasper

[Jasper St. Pierre]

Hi Geoff,

It seems binding a VAO with the correct vertex inputs fixed the issue. Thanks for the pointer to VertexExecutable, that got me pointed in the right direction.

--

  Jasper

[Geoff Lang]

Glad you figured it out.

By "change", I mean vertex input format (type parameter to glVertexAttrib*).  GL allows you to specify your vertex data with a bunch of different formats/types and not all of them work with a single D3D shader so we have to generate new vertex shaders.  Your best bet is to set up your vertex state the in the way you plan on using the program while linking it.

[Ken Russell]

Wow. Great diagnosis and collaboration.

Could we document this tip somewhere on https://www.khronos.org/webgl/wiki/Main_Page ? We should also make the workaround more visible; I think it's worth writing an email to public_webgl about using this pattern to avoid dynamic shader recompilation behind the scenes on Windows.

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/CA%2BPGBXuXyvQad0_FuMnAwn7-DG5Y6fS%2BeB65dUbqF3pesWPB%3DA%40mail.gmail.com.