Enable canvas low-latency mode on Mac

[keyou z]

When optimizing page performance using SharedWorker + OffscreenCanvas, it was discovered that the first 50 frames of the Canvas on macOS freeze and fail to refresh. This is the issue described here: https://issues.chromium.org/396460819.

The issue arises because in the first 50 frames, the CanvasResource is not sent to the Placeholder, keeping its reference count at 1. This causes the CanvasResourceProviderSharedImage to operate in non-copy-on-write mode, which uses the same SharedImage to update the Canvas content each time. This mechanism is used to enable the Canvas's low-latency mode (desynchronized=true), also known as SingleBuffer mode in CanvasResourceProvider. However, the non-copy-on-write mode currently doesn't work on Mac. Resolving this issue should allow Mac to support low-latency mode.

The non-copy-on-write mode requires a SharedImage to be both written by the Canvas and read by the viz compositor for display, may leading to the following issues:

1. If the SharedImage is written again before the viz compositor reads it, display correctness issues may occur.

2. If the SharedImage is directly used as an overlay, subsequent writes might be displayed without going through SubmitCompositorFrame. For example, on Mac, dragging a window can immediately reflect changes to the UI when using the same IOSurface.

3. On macOS with Metal enabled, the SharedImage used by the Canvas has an IOSurfaceImageBacking backend, which doesn't support concurrent read/write under Metal. However, non-copy-on-write heavily relies on concurrent read/write.

4. On macOS, most CompositorFrames can be displayed via the overlay mechanism. In this case, if the Canvas keeps drawing on the same IOSurface, the UI won't refresh.

If the user sets desynchronized=true, issues 1 and 2 can be considered acceptable. Therefore, the key to enabling non-copy-on-write on Mac lies in solving issues 3 and 4.

For issue 3:
The code can simply remove the blocking of concurrent read/write in IOSurfaceImageBackingFactory, but the issue https://anglebug.com/7626 needs to be re-evaluated.
For issue 4:
Some workarounds include setting CALayer.content to nil before each frame commit or creating a new CALayer to carry the IOSurface each time.

I have submitted a prototype CL to demonstrate potential solutions for issues 3 and 4: https://chromium-review.googlesource.com/c/chromium/src/+/6301417.

Thus, to make low-latency mode work on Mac, we need to discuss issues 3 and 4, or any other unknown factors.

Special thanks to Colin Blundell, without whose encouragement this email would not have been possible.

Thanks!

[Ken Russell]

Hi Keyou,

Thanks for your clear bug report, test case and investigation. I've commented on the bug and CC'd colleagues who know the SharedImage infrastructure better.

It seems to me that the focus should be on fixing OffscreenCanvas's behavior in SharedWorker. I would not recommend pursuing the desynchronized:true code paths and trying to make them work on macOS; that mode was added specifically for pen input on ChromeOS and can introduce visual tearing and undesirable behavior on that and some other platforms. Instead I think we should focus on making OffscreenCanvas work better in the regular, synchronized, mode on the main thread, dedicated workers, and SharedWorkers.

Thanks,

-Ken

[keyou z]

The focus is to enable canvas low-latency mode on Mac, not fixing OffscreenCanvas's behavior in SharedWorker, that issue is just the background of this question, sorry I didn't express it clearly.

Now `desynchronized:true` isn't just used for pen input, but a lot of other places as demonstrated in this blog, and it's now part of the web standards, so I think it's something that's well worth discussing in more depth.

Thanks!