Blend modes issue on Metal
260 views
Skip to first unread message
William Candillon
Feb 14, 2022, 4:03:53 PM2/14/22
to skia-discuss
Hello, Skia community!
We are maintaining a Skia binding for React Native binding.
On Android, we use OpenGL as a backend and on iOS we use Metal.
On iOS, blend modes are not working as expected:
```
auto canvas = skSurface->getCanvas();
auto r = 300;
SkPaint paint;
paint.setBlendMode(SkBlendMode::kMultiply);
SkPaint cyan(paint);
cyan.setColor(SK_ColorCYAN);
canvas->drawCircle(r, r, r, cyan);
SkPaint magenta(paint);
magenta.setColor(SK_ColorMAGENTA);
canvas->drawCircle(2 * r, r, r, magenta);
SkPaint yellow(paint);
yellow.setColor(SK_ColorYELLOW);
canvas->drawCircle(3 * r, r, r, yellow);
canvas->flush();
auto r = 300;
SkPaint paint;
paint.setBlendMode(SkBlendMode::kMultiply);
SkPaint cyan(paint);
cyan.setColor(SK_ColorCYAN);
canvas->drawCircle(r, r, r, cyan);
SkPaint magenta(paint);
magenta.setColor(SK_ColorMAGENTA);
canvas->drawCircle(2 * r, r, r, magenta);
SkPaint yellow(paint);
yellow.setColor(SK_ColorYELLOW);
canvas->drawCircle(3 * r, r, r, yellow);
canvas->flush();
```
This is the result on iOS: https://capture.dropbox.com/I6X0HggpQEdWrFUN?src=ss
This is the iOS code: https://github.com/Shopify/react-native-skia/blob/blendmode-bug/package/ios/RNSkia-iOS/RNSkDrawViewImpl.mm#L73
On iOS the backend is setup with kBGRA_8888_SkColorType is used whereas is it kRGBA_8888_SkColorType Android. Could that be the issue? The colors display properly, just the blending modes.
Are there configuration settings we could play with to try to pin down, maybe fix the issue? We would be so excited to get leads on to investigate on this.
Kind regards,
William
Brian Osman
Feb 14, 2022, 4:14:43 PM2/14/22
to skia-d...@googlegroups.com
That's pretty strange - and the results seem wrong in a way that's more severe than just swapping color channels (eg, BGRA vs. RGBA). Can you take a Metal frame capture? (https://developer.apple.com/documentation/metal/frame_capture_debugging_tools). That will probably give us the most information about what's happening.
For reference, I assume that the Android version of this produces output like: https://fiddle.skia.org/c/10af06521bf865c8dbd21aca09d56fd1
--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/0e7a7c63-abaa-471f-b77a-266acc70d840n%40googlegroups.com.
William Candillon
Feb 14, 2022, 4:29:05 PM2/14/22
to skia-discuss
Thank you for the reply!
This is the trace: https://www.dropbox.com/sh/wi4fwvdz3y5zmbb/AADhTbYDDxoUdgmxDVOaSN-1a?dl=0
Are there interesting I should be looking at?
Indeed on Android, we are getting the same result as on the fiddle.
Kind regards,
William
Greg Daniel
Feb 14, 2022, 4:41:21 PM2/14/22
to skia-discuss
Just to confirm, is Brian's fiddle correct in that the base colors of the circles are Cyan, Magenta, and Yellow? And that they are drawn in that order?
I'm asking because I'm really confused about the cyan/blue circle in the iOS case. Like if that is drawn first it should be blending just with a white background. So that fact that it is getting the color of multiply(Cyan, Magenta) everywhere is really weird. Like at first I was wondering if the error is related to just getting the dst color in one place and using it for all the pixels instead of sampling each pixel separately. But that doesn't make sense for the first draw.
What happens if you only draw the Cyan circle?
I'm asking because I'm really confused about the cyan/blue circle in the iOS case. Like if that is drawn first it should be blending just with a white background. So that fact that it is getting the color of multiply(Cyan, Magenta) everywhere is really weird. Like at first I was wondering if the error is related to just getting the dst color in one place and using it for all the pixels instead of sampling each pixel separately. But that doesn't make sense for the first draw.
What happens if you only draw the Cyan circle?
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/4d310b3d-0011-4b0b-a7da-cb92d8dc6339n%40googlegroups.com.
William Candillon
Feb 14, 2022, 4:47:48 PM2/14/22
to skia-d...@googlegroups.com
Indeed, with blendmode multiply the result for any color is the color
multiplied with the color magenta.
This seems to be consistent with another iOS bug we have where any
backdrop filter returns just magenta (image filters work great on iOS
but not when used as background filter, we assume this is because
backdrop filter are using blending).
On Android the example is pixel perfect with the fiddle.
On iOS this is just the cyan circle:
https://capture.dropbox.com/TjtOa18HEt31ysuS?src=ss (blue)
I've draw a green circle expecting it to be black (magenta * green):
https://capture.dropbox.com/vrsqopMWRd3DJdLI?src=ss (black)
Kind regards,
William
> You received this message because you are subscribed to a topic in the Google Groups "skia-discuss" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/skia-discuss/FTO0NSR_09Q/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to skia-discuss...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAL_WtP6QkK1y%2BnWEwdXfNzM0i6g40raaKsaeTRC%2BXb7acEX0bQ%40mail.gmail.com.
multiplied with the color magenta.
This seems to be consistent with another iOS bug we have where any
backdrop filter returns just magenta (image filters work great on iOS
but not when used as background filter, we assume this is because
backdrop filter are using blending).
On Android the example is pixel perfect with the fiddle.
On iOS this is just the cyan circle:
https://capture.dropbox.com/TjtOa18HEt31ysuS?src=ss (blue)
I've draw a green circle expecting it to be black (magenta * green):
https://capture.dropbox.com/vrsqopMWRd3DJdLI?src=ss (black)
Kind regards,
William
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/skia-discuss/FTO0NSR_09Q/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to skia-discuss...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAL_WtP6QkK1y%2BnWEwdXfNzM0i6g40raaKsaeTRC%2BXb7acEX0bQ%40mail.gmail.com.
William Candillon
Feb 14, 2022, 5:01:09 PM2/14/22
to skia-d...@googlegroups.com
I also tried on iOS simulator on an intel laptop (previously we used
arm64) and the colors are different:
https://www.dropbox.com/s/yhrqgfsi3lb6ioa/Screen%20Shot%202022-02-14%20at%2022.57.11.png?dl=0
I am also getting different colors on my physical device (my partner
Christian however is getting the same result between the simulator and
the physical device).
Kind regards,
William
arm64) and the colors are different:
https://www.dropbox.com/s/yhrqgfsi3lb6ioa/Screen%20Shot%202022-02-14%20at%2022.57.11.png?dl=0
I am also getting different colors on my physical device (my partner
Christian however is getting the same result between the simulator and
the physical device).
Kind regards,
William
Greg Daniel
Feb 14, 2022, 5:04:40 PM2/14/22
to skia-discuss
Okay looking at the trace the texture we are sampling from which is supposed to represent the dest is solid magenta. So the bug is in how we are copying the main color attachment to an offscreen texture. Or more specifically how we are NOT copying the texture. I would expect to see a blit encoder in the trace you sent but there is none. I think the magenta may be metal iOS's default uninitialized data color.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAM7ytN_pbQ9kqbqGa8ho1ZoT-jF4Vq7e%2BOb_KJF69_HrQPmDaQ%40mail.gmail.com.
William Candillon
Feb 14, 2022, 5:10:32 PM2/14/22
to skia-d...@googlegroups.com
Thank you so much for taking the time to look into this.
Could it be because of how we initialize the Metal backend?
I assume that the issue doesn't exist on Flutter or Chrome with Metal?
I looked at flutter/engine to see how the backend is created:
https://github.com/flutter/engine/blob/e444009bf46b39af0b2a2aa6e7c28c668c50004c/shell/platform/darwin/graphics/FlutterDarwinExternalTextureMetal.mm
I didn't see any big difference but my level of expertise with this is
so low that I might be missing something.
This is where we get the surface from the backend:
https://github.com/Shopify/react-native-skia/blob/blendmode-bug/package/ios/RNSkia-iOS/RNSkDrawViewImpl.mm#L61
Another thing we tried is on iOS, is to create an offscreen surface,
run the drawing there and take a screenshot of it. Here the
screenshots has the correct colors (like on Android).
Kind regards,
William
On Mon, Feb 14, 2022 at 11:04 PM 'Greg Daniel' via skia-discuss
> To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAL_WtP5x19_9yOd%2B%3Daew-mi6466LfzmCjeEG5VJ0Jt2Xngz4zg%40mail.gmail.com.
Could it be because of how we initialize the Metal backend?
I assume that the issue doesn't exist on Flutter or Chrome with Metal?
I looked at flutter/engine to see how the backend is created:
https://github.com/flutter/engine/blob/e444009bf46b39af0b2a2aa6e7c28c668c50004c/shell/platform/darwin/graphics/FlutterDarwinExternalTextureMetal.mm
I didn't see any big difference but my level of expertise with this is
so low that I might be missing something.
This is where we get the surface from the backend:
https://github.com/Shopify/react-native-skia/blob/blendmode-bug/package/ios/RNSkia-iOS/RNSkDrawViewImpl.mm#L61
Another thing we tried is on iOS, is to create an offscreen surface,
run the drawing there and take a screenshot of it. Here the
screenshots has the correct colors (like on Android).
Kind regards,
William
On Mon, Feb 14, 2022 at 11:04 PM 'Greg Daniel' via skia-discuss
Greg Daniel
Feb 14, 2022, 5:19:53 PM2/14/22
to skia-discuss
Is it possible that your mtl texture is being created from the layer with the FramebufferOnly flag set? I'm trying to figure out how I can see this information in the trace debugger but haven't found it yet.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAM7ytN-K8i4SxaNiwzeEYKRLF5AoYY8N%3DsoP%2BUA8sXpXmOX8Og%40mail.gmail.com.
Greg Daniel
Feb 14, 2022, 5:23:55 PM2/14/22
to skia-discuss
Specifically asking about this, https://developer.apple.com/documentation/quartzcore/cametallayer/1478168-framebufferonly?language=objc on the CaMetalLayer
Jim Van Verth
Feb 14, 2022, 5:26:51 PM2/14/22
to skia-d...@googlegroups.com
In my local test, setting layer.framebufferOnly = NO; fixed it.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAL_WtP63EjwVfazi6dfMg72S18Hu3nyJthSjXGWS8aVdtomfaA%40mail.gmail.com.
William Candillon
Feb 14, 2022, 5:33:52 PM2/14/22
to skia-d...@googlegroups.com
I can confirm that this fixed the issue on my side too! A huge thank you.
I'm incredibly grateful for such quality support.
You guys are amazing.
Kind regards,
William
> To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAM5o3G2ynMA2kR2ktA8FhtA9PwkFmmNZ%2BbiAU0vELAhpR_czHA%40mail.gmail.com.
I'm incredibly grateful for such quality support.
You guys are amazing.
Kind regards,
William
Greg Daniel
Feb 14, 2022, 5:34:06 PM2/14/22
to skia-discuss
So I think then the "correct" behavior here is that we should have just dropped the draws all together since we cannot do the blending with framebufferOnly objects. I think we are missing a check somewhere in this function: https://source.chromium.org/chromium/chromium/src/+/main:third_party/skia/src/gpu/v1/SurfaceDrawContext.cpp;l=2036.
Probably need some line like: if (this->asSurfaceProxy()->framebufferOnly) { return false; }
Probably need some line like: if (this->asSurfaceProxy()->framebufferOnly) { return false; }
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CAM5o3G2ynMA2kR2ktA8FhtA9PwkFmmNZ%2BbiAU0vELAhpR_czHA%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages