Questions on Skia Blur algorithm

[William Candillon]

I investigated implementing the iOS variable blur filter with Skia.

You can see a demo at https://twitter.com/wcandillon/status/1695528007245398053/video/1

To implement such an effect, we need shaders to implement the blur, and for each xy value, depending on the alpha value of the mask, we pick a variable blur amount.

I've tried different blur shaders, which have different tradeoffs.

I used this two-pass blur example: https://lisyarus.github.io/blog/graphics/2023/02/24/blur-coefficients-generator.html, which "looks" somewhat similar to what Skia does. But the quality of the results is nowhere near what Skia provides.

To implement the variable blur, I am making an assumption that I would like to double-check: I'm taking the offsets and kernels for a sigma of 1 and for a sigma of X (maximum blur) and do the linear interpolation of these values to get a blur amount between 1 and X. Is this correct?

 I looked at the GLSL version of the blur shaders generated by Skia to reverse engineer as a Skia Shader. 

Is there a public Skia API that would allow us to generate the kernel and offset values directly from C++? We would aim to use these values as uniform in our custom blur shader. Or could I use direct values generated from https://lisyarus.github.io/blog/graphics/2023/02/24/blur-coefficients-generator.html?

Let me know if you have any leads for me to continue investigating this topic.

Kind regards,

William

[John Stiles]

Skia's BlurUtils.cpp (e.g. Compute2DBlurKernel) is responsible for building the kernel/offset data for a blur, but it's not public API. 

Offhand I am not sure if lerping the kernel/offset values between two different blurs is going to be close enough to ground truth to look right. I'm pretty sure it's not mathematically/rigorously correct, but visually it might be pretty good. 

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/6e808928-6833-4fc6-bf46-41c65c5b5966n%40googlegroups.com.

[William Candillon]

Thank you for the insight and I agree about the lerp. I guess the next step would be to investigate how to calculate the kernel within the shader.

Kind regards,

William

You received this message because you are subscribed to a topic in the Google Groups "skia-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/skia-discuss/mL2iaiwulmc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CANBPXuB%3D5arw-dw9s2%3DBGkaPbt8z_oNZH1TDAqA5awGPEyL6fQ%40mail.gmail.com.

[Michael Ludwig]

The blur utilities have recently been shuffled around and are now in src/gpu/BlurUtils instead of src/gpu/ganesh/GrBlurUtils. However, these are not functions that we'd want to have in the public API. Since we're open source, though, you can definitely look at them to see how we are calculating the values. The Compute2DBlurKernel(https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/gpu/BlurUtils.cpp;bpv=1;bpt=1;l=18?) function actually computes the full kernel assuming you're doing a pixel sampler for each element in the convolution matrix (i.e. no linear filtering optimization).  This is used to calculate ground truth values that we can use in the 1D linear-filtered passes and it's also used directly when the blur sigma is small enough that a heuristic says it's better to do more texture samples within one pass than pay the cost of render pass switches on the GPU.

The function Compute1DBlurLinearKernel (https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/gpu/BlurUtils.cpp;l=76) shows how to derive the sampling offsets and adjusted kernel weights from the original full 1D kernel.  One thing to note is that if you're using the linear filtering optimization and your offsets are being added to the `coord` passed into the SkShader, you need to carefully control that these coords are 1-to-1 with the actual fragment coords (e.g. your canvas matrix is just the identity or integer translation). Otherwise the actual coordinate passed to the input of the blur will end up with texture coords that don't work out to the intended weighting for the 1D blur.

Lastly, another piece of SKia's blurs is that when the sigmas are large (currently > 4, but we can likely push this up to 9 pretty soon), we will do progressive downsampling so we can evaluate a lower resolution image, and then we upscale that blurred image to simulate the larger blur. For variable blurs that won't be an option.

I haven't worked through the math yet, but given that the kernel weights are calculated with functions that put the sigma (function input) into the exponent *and* there's a normalization step that sums the neighboring weights for the denominator, it seems highly likely that the weight function is not linear in terms of sigma, so linearly interpolating them will not produce exact results. I would recommend playing around on desmos or with your demo to see if linear interpolation is acceptable, or if you can get away with a piecewise linear interpolation between several pre-defined sigma values.

[John Stiles]

If the goal is to get a visually pleasing blur with pixel-by-pixel control over the blur level, one approach I've seen used is to blur the image to a handful of different levels (e.g. 4 images is probably enough), then at each pixel, lerp between adjacent blur-level images to approximate the pixel blur level. e.g.:

Blur  0 ~ 0.25: mix(originalImage, blurLevel1)

Blur  0.25 ~ 0.5: mix(blurLevel1, blurLevel2)

Blur  0.5 ~ 0.75: mix(blurLevel2, blurLevel3)

Blur  0.75 ~ 1.0: mix(blurLevel3, blurLevel4)

You can experiment with the number of levels needed to get a convincing effect. If it's a quick transition you can probably cheat heavily here. :)

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/1c34bb25-65b0-4f30-896a-333e74c803d9n%40googlegroups.com.

[William Candillon]

Thank you for the suggested solution, John; it seems to work really well (of course, if the blur delta between each step is not too big).

[William Candillon]

Thanks to this discussion, I found a good implementation of the effect with SKSL.

You can see a demo here: https://twitter.com/wcandillon/status/1742222953918447912

The implementation: https://github.com/wcandillon/can-it-be-done-in-react-native/blob/master/bonuses/skia-examples/src/BlurGradient/BlurMask/Shader.ts#L5

I noticed that in the default blur image filter in Skia, there is some random/seed value, which you can notice when animating the blurred view (or maybe the way clamping is done).

It's slightly noticeable in this example: https://twitter.com/wcandillon/status/1733972374930837776 (you can see a slight jitter in the background). This doesn't seem to happen with our custom blur effect.

Kind regards,

William