Drawing is warped during window resize on NVIDIA graphics card

[bmitc]

I have filed this issue on SkiaSharp: https://github.com/mono/SkiaSharp/issues/2832

However, it is not clear whether the bug is in SkiaSharp, Skia itself, the backend graphics API Skia is using (OpenGL in this case), or NVIDIA drivers.

Does anyone have any suggestions on what to do about this? This is a major limitation I am experiencing right now, and I just do not have the experience or knowledge on where to even report it, much less fix or workaround it. I've been trying for around a year to find a good workaround but haven't. Even disabling resizes on the window while handling the resize callback does not prevent this issue.

Thank you for any help!

[John Stiles]

I'm pretty sure this is a GLFW issue. Skia has no concept of window resizing, sorry. 

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/4c50c78a-bd30-404d-8267-a2b1f0d0447en%40googlegroups.com.

[Blake Mitchell]

Thank you for taking a look. However, that's what the GLFW folks said. :) They said it likely wasn't a GLFW issue demonstrated by the fact that drawing done by pure OpenGL commands and not SkiaSharp/Skia do not demonstrate this issue. The explanation was also that GLFW does almost nothing besides forward things to the underlying OS libraries. I verified the GLFW boing example (https://github.com/glfw/glfw/blob/master/examples/boing.c) still doesn't exhibit this issue. (Note: One must place `display(); glfwSwapBuffers(window);` at the end of `reshape`.)

See here for when I originally reported this to the GLFW forum: https://discourse.glfw.org/t/canvas-jitter-flicker-when-resizing-with-multithreaded-render-loop/2188

At some point, there was a regression, somewhere in the stack, because now this issue shows up even in the most simple, single-threaded case. (I was never convinced my threading code was wrong anyway, as it's very hard to get wrong in F#.) I am unable to get back to where this never happens in the single-threaded case, which makes me believe it's possibly a graphics driver issue or an issue uncovered by a graphics driver change.

I just don't know what to do. If the issue was in my reproducing example, I figure someone would have pointed it out already, as it's very simple. And it isn't clear how to resolve it when every part of the stack declares it a problem of some other part of the stack. While Skia or SkiaSharp may not have a concept of resizing themselves, they have a concept of drawing, and it seems plausible to me that if it's drawing to the wrong thing (surface/canvas) then this could happen. Is that possible?

Are there any suggestions on how to whittle this down to a specific part of the stack?

You received this message because you are subscribed to a topic in the Google Groups "skia-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/skia-discuss/UEkA-DPcC9M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CANBPXuDaVY%3DX1-UeOY7itr2X%2BFMZS9DJV7g9_vXj8tbZ5frK7Q%40mail.gmail.com.

[John Stiles]

I took a look at the working example that you posted (https://github.com/glfw/glfw/blob/master/examples/boing.c) and it's pretty different from your non-working code.

Specifically, I'm looking at the callback function for glfwSetFramebufferSizeCallback.

The working code uses a function called `reshape`—note that the only thing reshape does is rendering calls to repaint the image.

Your code uses a function called `draw`. In addition to repainting, `draw` also calls glfw.PollEvents() and glfw.SwapBuffers(window). 

It's just a guess, but I highly suspect that these extra calls are related to your issue. 

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAPxUcnGY3N2hrvdW%3DX217AqyQO21SAj40HWbdsBZhp0cGxdGHQ%40mail.gmail.com.

[Brian Osman]

John's theory makes sense. From looking at the repro videos, it REALLY looks like it's not "bad scaling when drawing". Rather: it's swapping the wrong buffer, possibly by swapping too many times (thus re-presenting an old buffer that was rendered when the window was a different size, therefore causing it to look stretched).

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CANBPXuB%3Do3RGtLtvu2CYVRJ_qQUbV7SNXD%2BviC%2Buavb_U9157Q%40mail.gmail.com.

[Blake Mitchell]

Hi Brian and John,

Thank you both for taking the time to take a look! It is very much appreciated. The extra `glfw.PollEvents()` was indeed a mistake in my example, as it should only be called in the main loop. However, moving it to the correct place (out of `draw` and just as a statement in the loop) makes no difference in the behavior.

In my previous comment, I mentioned that one needs to place `display(); glfwSwapBuffers(window);` at the end of `reshape` in the `boing.c` example to get the equivalent behavior I'm wanting, that is that the window redraws itself while it is being resized. Without adding those two commands, the `boing.c` example just stretches the "frozen" scene during a resize until the resize stops, at which point it will draw it correctly. Adding these two commands to `boing.c` does not reproduce the issue shown in the video with SkiaSharp, so I don't think it's the swap buffers. Swapping buffers is required because once you draw to the canvas, it has to be swapped to the front so that it's actually displayed during the resize.

The fact that this only occurs with NVIDIA graphics and not integrated Intel graphics is also something not explained by this.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAMcBjo5r7fkgdPdeD3ECHqQ2XP_N1zYL39Q9Se5-9wj7Wn42Eg%40mail.gmail.com.

[Blake Mitchell]

Here's an interesting experiment: https://imgur.com/a/B6qwcyR

If I just flat out disable the resize callback, I note that there are indeed still differences between the integrated Intel graphics and NVIDIA graphics that I do not see with the `boing.c` example (which is just GLFW and OpenGL in C++). With GLFW and SkiaSharp, when resizing with no callback with integrated Intel graphics, the display remains completely static through the resize (other than the black areas). When resizing with no callback with NVIDIA graphics, the display stretches or compresses during the resize. It does so in the exact same way seen above when there's a resize callback, it's just that there, the resize callback seems to be correcting this warping after the fact.

So it seems to me, the order of operations in the NVIDIA situation could be:

1. A resize event occurs
2. Before the framebuffer resize callback occurs, something (either GLFW, the OS window API, SkiaSharp, Skia, OpenGL, or NVIDIA driver) is allowing the resize to occur or at least the framebuffer thinks it's being resized.
3. The image warps due to (2)
4. The framebuffer resize callback is called, which corrects the image to the correct image
5. Rinse and repeat during a resize

[John Stiles]

If you are not calling into SkiaSharp at all during the window resize, then I don't see how it's possible that Skia or SkiaSharp could be related to this behavioral change. You've taken Skia out of the equation, which is actually useful since it helps narrow down suspects.

As an experiment, have you tried calling `glViewport(0, 0, width, height)` at the start of your callback? It looks like most code which implements a framebuffer callback does this immediately, and it seems like an easy thing to try.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAPxUcnHUjAcvAKg-TRcYGVY%2B3w7r_ubj0DzMrQQm2QW-wkFPxg%40mail.gmail.com.

[Blake Mitchell]

Thanks again, John.

I've been doing several experiments, so apologies for the late response. I have incorporated setting the viewport size by using the Silk.NET.OpenGL bindings in both the framebuffer resize callback and the main draw loop. It seems to have a stabilizing effect on the resize jitter, but it does not completely remove it. This is particularly when I have the lower-right corner of the window dragged near the edge of the monitor. It's currently not clear how to properly call this in the cases where the rendering is on a separate thread.

A question for the Skia folks is regarding this resize behavior. It was my assumption (perhaps incorrect) that all of this was taken care of by recreating the Skia surface. For example:

if surface <> null then surface.Dispose()

if grBackendRenderTarget <> null then grBackendRenderTarget.Dispose()

grBackendRenderTarget <- new GRBackendRenderTarget(width, height, 1, 0, grGlFramebufferInfo)

surface <- SKSurface.Create(grContext, grBackendRenderTarget, GRSurfaceOrigin.BottomLeft, SKColorType.Rgba8888)

canvas <- surface.Canvas

Does the Skia surface not set the viewport underneath the hood? I have tried inspecting both the SkiaSharp and Skia codebases, but I must admit that I had trouble navigating and answering this question myself. Where I understood that the surface needed to be recreated every resize frame was from here: Correct way for resizing the SkSurface when the GL viewport changes (google.com), which was a question asked by the SkiaSharp author several years ago.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CANBPXuBoUPKhH2PGzOVmxhSTV-EN0sz_UGN%2Ba3N4pzKhDMHOVw%40mail.gmail.com.