SkSL optimizer testing

91 views
Skip to first unread message

Noah Lavine

unread,
Dec 4, 2019, 4:12:06 PM12/4/19
to skia-d...@googlegroups.com
Hi Skia folks,

I'm working on a small patch to improve the SkSL optimizer. As part of that project, I've been looking into how to test the optimizer. Are there currently optimizer tests? I couldn't find any in tests/.

The testing approach I've been working on is converting two programs to SkSL::Program objects, using the compiler to optimize both, and then testing if the resulting IR is equivalent. For instance, I would test constant folding for addition by checking that the following compile to equivalent IR:
  1. "void main(inout half4 color) { color.r = 0.25 + 0.25; }"
  2. "void main(inout half4 color) { color.r = 0.5; }"

The problem I've run into is checking whether two IR programs are equivalent in the sense I need. I believe it's doable, but doing it right would be a relatively intrusive change that would touch every IRNode subclass. Before I start on a change like that, I'd like to check with the development team for feedback on the general approach.

Another option I considered would be to not test the optimizer directly, but just verify that programs have the expected output, like SkSLInterpreterTest does. That would require less up-front work, but I don't think it tests the right thing, since the optimizer could do nothing and the program would still have the correct output.

A third option would be to (again) not test the optimizer directly, but observe the performance difference in timing runs before and after an optimizer change. That would work, but I think that a more fine-grained testing approach would make it easier to diagnose problems if the optimizer stops working in the future.

And please let me know if I'm completely off-base with this question - I'm still trying to get up to speed on the codebase.

To summarize, are there already standard ways of testing the SkSL optimizer that I haven't found? And if not, would you be interested in a patch to add optimizer testing by checking IR equivalence?

Thanks,
Noah Lavine

Mike Klein

unread,
Dec 4, 2019, 4:19:41 PM12/4/19
to skia-discuss
That sounds cool to me, though you probably want to test that optimized(A) == unoptimized(B) if possible, right?  That rules out two bugs that accidentally hide each other.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CA%2BU71%3DO8KoDgE%3DOhWiQwV%3D6Gd7SBLHr%2B_cfAKd71ZosP0X6e%3DQ%40mail.gmail.com.

Mike Klein

unread,
Dec 4, 2019, 4:21:08 PM12/4/19
to skia-discuss
Not that I'm the authority.  That'd be Ethan I'd say.  I literally just think it sounds cool.

Ethan Nicholas

unread,
Dec 4, 2019, 4:29:53 PM12/4/19
to skia-d...@googlegroups.com
The approach so far has been to look at the GLSL output. There aren't really any tests *only* of the optimizer, but there are many tests that ensure that the GLSL output contains the expected optimizations. If you disable the optimizer (try adding a "return true" right after program.fIsOptimized = true in SkSLCompiler.cpp), you will see that many of the tests is SkSLGLSLTest.cpp will fail as a result.

There are certainly other ways we could have chosen to test the optimizer, but since the GLSL output is a near-verbatim dump of the IR, this seemed like the simplest approach. And I would absolutely welcome any further optimizer-specific tests and improvements!

--

Mike Klein

unread,
Dec 4, 2019, 4:35:57 PM12/4/19
to skia-discuss
Ah glad you just said that... I had the same though just now but to dump to SPIRV.

Do you have any optimizations in mind that we can do that the final shader compiler can't?  Gotta figure they're folding 0.25+0.25 already right?

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAFqx8w%2BbOL%2BzbrQY4H3iEek%3DSHABUGcpGUr_Bkggf1XX8TH6Uw%40mail.gmail.com.

Ethan Nicholas

unread,
Dec 5, 2019, 9:38:04 AM12/5/19
to skia-d...@googlegroups.com
I honestly don't know what optimizations the drivers typically perform. We handle many of the basic ones like constant folding and constant propagation ourselves; they're cheap to implement, can't really hurt, and make static if statements more useful.

We don't currently bother with more expensive or difficult optimizations like strength reduction and loop unrolling. My concerns with the bigger optimizations are that we already have people complaining that the shader compilation step takes too long, so I'm leery of adding even more overhead, and the backend might perform the optimization itself as well. At best, that means we'd be wasting time duplicating effort, and at worst, our optimizations might screw up other optimizations performed by the backend and end up slowing the shader down rather than speeding it up.

Brian Osman

unread,
Dec 5, 2019, 10:18:06 AM12/5/19
to skia-d...@googlegroups.com
My only concrete knowledge is from years ago, when many mobile drivers did little or no GLSL optimization. Hopefully that's improved, but I don't know if anyone has confirmed that recently. For example, Unity maintained a project called glsl-optimizer for a long time (and it was used by many other game companies), to work around this fact:  https://github.com/aras-p/glsl-optimizer.

There are also some optimizations that drivers may not feel comfortable doing, but are safe over many/most inputs. Many, many examples in Emil Persson's fantastic 2013 GDC Talk: http://www.humus.name/index.php?page=Articles&ID=6

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAFqx8w%2BhSkAyyOxpKoNpp2DwtvY1PkEr%3DOJzu2d0nvNThrx4Lw%40mail.gmail.com.

Mike Klein

unread,
Dec 5, 2019, 10:28:53 AM12/5/19
to skia-discuss
Yeah, I've been curious about both how far shader compilers are permitted and are willing to transform operations on floats as if they were operations on reals.  Is x*0=0?

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAMcBjo69htvGzPHKayi9rW4%2BE9O2Wuyj24s5BUVNvJjohEP_Tg%40mail.gmail.com.

Ethan Nicholas

unread,
Dec 5, 2019, 11:13:26 AM12/5/19
to skia-d...@googlegroups.com
IIRC shaders are not required to follow the full IEEE spec, so they can do things like assume x * 0 = 0 and treat x / 0 as undefined behavior.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAD%3DHvMsmvZMPjWfhrGC3ajF3qr0T2HbmtU47Ui1k_QpWj3i%3D8g%40mail.gmail.com.

Mike Klein

unread,
Dec 5, 2019, 11:35:05 AM12/5/19
to skia-discuss
What rules do they have to follow?

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAFqx8wLGkENJ1t9-B4r%2B7DosTy1-ZLfEZCAOumax2REQktE%3D%3Dg%40mail.gmail.com.

Noah Lavine

unread,
Dec 5, 2019, 3:23:02 PM12/5/19
to skia-discuss
Hi Mike, Ethan and Brian,

Thanks for your replies! Brian, that is a fantastic presentation, thank you for the link.

About the SkSL optimizer, I will dump to GLSL for testing. Thank you! Mike, I see your point about my approach hiding bugs. I think the GLSL tests are roughly equivalent to what you suggested. I was trying to find a way to directly test whether "you can write code either of these ways, it doesn't matter which" holds, which I think is what I care about as a shader author. I think the tradeoff is that comparing to either unoptimized IR or GLSL lets you catch some more bugs, at the cost of having to update your tests when you update your optimizer.

Ethan, thank you for all of the information! I was just planning to see if I could add a little more constant folding. It's partly intended to be useful and partly a way to learn more about Skia. However, I see your concern about making the compilation step take too long. It looks like bench/SkSLBench.cpp measures the SkSL compiler speed. Is that a good place to look for the impact a new optimization would have on compiler speed? Do you have more examples of "typical" shaders whose compilation speed you care about?

Thanks,
Noah
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-d...@googlegroups.com.

Noah Lavine

unread,
Dec 9, 2019, 1:12:53 PM12/9/19
to skia-d...@googlegroups.com
Hi Ethan,

Please take a look at https://skia-review.googlesource.com/c/skia/+/258877 . It's a small patch, so I hope it will be a quick review.

Thanks,
Noah

To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/98fe33fe-b400-4647-994e-81f9f1d3ca96%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages