Graphite on iOS

[orto...@gmail.com]

Hello,

I'm trying to switch to the new graphite from ganesh. I had it working correctly (drawing SVG with changing paths) on iOS using CAMetalLayer. I've been trying to port it to graphite, using GraphiteMetalWindowContext.mm as example. I got it working but it's behaving strangely.

First, it looks like graphics context is not always initialized correctly, as I can't see the GPU stats overlay from ios (even if the app is rendering). Then if I change a path, the overlay appears but app starts to consume a lot of memory and crash. 

Sometimes at app start, the screen goes pink with some errors

Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (00000004:kIOGPUCommandBufferCallbackErrorSubmissionsIgnored)

Is there a minimal graphite example for ios/android that I missed?

[Greg Daniel]

First to confirm, are you using a fairly recent version of the Graphite code?

We haven't tried running Graphite on iOS yet so I can't help directly there. So as you've probably found the main related windowing files we have for our test tools are GraphiteMetalWindowContext.mm and GraphiteMetalWindowContext_mac.mm. For our Viewer test app these do work on Macs running Graphite Metal. For iOS, I would look at the similar Ganesh files MetalWindowContext_mac.mm and MetalWindowContext_ios.mm to see what the differences are.

Android Graphite Vulkan viewer should also run fine on our viewer app and you can look at the code for that in the WindowContexts as well. Note there are some bugs/missing features currently in the Vulkan backend so some things may not draw correctly. But the windowing code should all work.

--
You received this message because you are subscribed to the Google Groups "skia-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skia-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/1a77f50b-8552-4d9f-912e-70609166d897n%40googlegroups.com.

[orto...@gmail.com]

I'm running from main branch, also tried m123.

I've tested the mac viewer app and it does indeed work as expected. On iOS is has issues, for example I can't capture and profile Metal frame, the app crashes as that happens.

Are there any plans to validate Graphite on iOS?

Some more details about the issue:

As I mentioned in the first post, the Metal HUD is not visible on the screen, and when it does appear randomly the app is killed for using too much memory. Sometimes appears for 0.1 seconds and dissappears, but the app continues to work normally when this happens.

I found that commenting the line fGraphiteContext->insertRecording(info); makes the HUD appear and stay as it would be normal, but the drawing does not work, not sure if that's helpful info.

[Jim Van Verth]

I can take a look next week -- I'm out of town and don't have a device to test on at the moment.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/677043da-80dd-43a7-9ec7-87bf8aecc620n%40googlegroups.com.

--

Jim Van Verth | Software Engineer | Google.com

[orto...@gmail.com]

Thank you. I've migrated my android code to graphite and it's working properly there, only ios having issues.

[Jim Van Verth]

With https://skia-review.googlesource.com/c/skia/+/828539 I was able to get our Viewer app up and running on iOS with Graphite. I can capture frames, and have the stats overlay running. This was with XCode 15.3 and iOS 17.4.

One thing that's different between the MetalWindowContext and the GraphiteMetalWindowContext is that framebufferOnly is set to FALSE in the layer. That's because Graphite depends on being able to read the render target for some blend types, and with framebufferOnly set to TRUE we can't do that with the layer's texture. I don't think this is your problem but it's something to be aware of.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/e37fd288-d61a-4106-ae32-ced84870ba03n%40googlegroups.com.

[orto...@gmail.com]

I tried to apply these changes to main branch and run viewer app on ios device but getting a crash (caused by first layer visit in Window::visitLayers, if I start it from 1 it will work including metal HUD but no drawing).

Is there a proper way to change context to graphite (I changed backend_type_for_window to return kGraphiteMetal_BackendType), or any special build args?

I've changed framebufferOnly property in my app but that did not help.

[John Stiles]

We actually saw a similar crash here, but it disappeared/wasn't reproducible after doing a full clean/rebuild. 

Our hunch was that there was some sort of version mismatch or issue with libc++, since it is crashing inside std::stringstream (and the Skia code here seems correct and has not meaningfully changed in a very long time).

Can you please try a clean rebuild from scratch? Or if you can figure out a way to make this issue reproducible, we'd love to dig into it further.

Thanks.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/655cfbaa-1c9d-4f5d-a3ad-e2061d1364f9n%40googlegroups.com.

[John Stiles]

One more hunch. Apparently Xcode 15's linker seems to cause crashes inside libc++ with sufficiently old iOS versions. You can try switching back to the old linker: https://forums.developer.apple.com/forums/thread/742470

Sorry, we aren't running Xcode 15 in our CI environment yet, so we wouldn't be able to catch Xcode 15-specific problems. (For that matter, Apple must also have a gap in their coverage matrix somewhere…)

[Jim Van Verth]

So I've tested this with two different XCode 15.3 setups. Originally I was hitting the possible linker issue with one of them, but now it appears to be gone. The only changes I made were to update my device to the latest OS, and to go into ~/Library/Developer/Xcode/iOS\ DeviceSupport/, delete everything in there, and then resync my device. I've verified this with an iPad running 17.4.1 and an iPhone 6s running 15.8.2 (the highest it can handle).

I'm also working on a CL to update our deployment setup (https://skia-review.googlesource.com/c/skia/+/827847) but I don't think that would affect you.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/CANBPXuA9qAijR36k__tBp5qvbEtAdENHSYBJgjCqeGtQSEQ0Fw%40mail.gmail.com.

[orto...@gmail.com]

I tried to delete the out folder and rebuild but still getting that crash. Using Xcode 15.3 with iOS 17.4.

Here I attached a minimal xcode project to initialize a graphite context. skia-test folder should be right next to skia folder. It also assumes library is build in skia/out/ios64.

You should be able to observe metal hud not appearing or intermitently appear. Sometimes also GPU draw errors.

Thanks

[John Stiles]

I'm not sure I understand. 

You're "still getting that crash" (which occurs at startup) but also have attached a project which apparently runs and draws, but sometimes shows draw errors/HUD is intermittent.

Based on the "running and drawing stuff" part, it sounds like the crash is actually gone?

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/5d1804c5-18ae-4d2c-b8c3-99a5da4b7516n%40googlegroups.com.

[orto...@gmail.com]

Sorry for the confusion. The crash is (still, will try to update to 17.4.1) only happening in the skia viewer app. I was giving viewer a try to see if it still has the issues im seeing in my application and potentially identify the cause.

The  project I attached showcases the issues I'm seeing in my application, which is why I opened this discussion here.

[orto...@gmail.com]

I'm still facing the crash in the viewer app. I found that skia is able to generate cmake build files, and then cmake can generate xcode project that will actually get compiled by xcode not ninja (passing --ide=xcode to gn still uses ninja it's not compiled directly).

Have run bin/gn gen out/config --args="is_debug=true target_os=\"ios\"" --ide=json --json-ide-script=../../gn/gn_to_cmake.py && cmake -GXcode.

There are many errors like this one when trying to generate project for ios, mac seems to succeed but did not see viewer app in the list.

CMake Error at CMakeLists.ext:208 (add_custom_target):
  Cannot find source file:

    /Users/user/skia/out/config/gen/blob_cache_sim

Is there another way to generate a xcode project that will actually build from source without ninja?

[orto...@gmail.com]

I was able to build skia lib using xcode buildsystem by generating xcodeproj with cmake. It only works for official build.

It did not help with the issue I'm facing, still getting errors in the simple app I'm using. Have tried arc on/off  but it made no difference.

Execution of the command buffer was aborted due to an error during execution. Caused GPU Timeout Error (00000002:kIOGPUCommandBufferCallbackErrorTimeout)
Execution of the command buffer was aborted due to an error during execution. Discarded (victim of GPU error/recovery) (00000005:kIOGPUCommandBufferCallbackErrorInnocentVictim)
Execution of the command buffer was aborted due to an error during execution. Discarded (victim of GPU error/recovery) (00000005:kIOGPUCommandBufferCallbackErrorInnocentVictim)
Execution of the command buffer was aborted due to an error during execution. Caused GPU Timeout Error (00000002:kIOGPUCommandBufferCallbackErrorTimeout)

Args used to build skia:

is_official_build=true \
  target_os=\"ios\" \
  target_cpu=\"arm64\" \
  ios_min_target=\"12.0\" \
  is_debug=false \
  skia_use_metal=true \
  skia_enable_metal_debug_info=true \
  skia_enable_graphite=true \
  skia_enable_pdf=false \
  skia_enable_skottie=false \
  skia_use_expat=false \
  skia_use_libjpeg_turbo_decode=false \
  skia_use_libjpeg_turbo_encode=false \
  skia_use_libpng_decode = false \
  skia_use_libpng_encode = false \
  skia_use_libwebp_decode = false \
  skia_use_libwebp_encode = false \
  skia_use_zlib = false \
  skia_use_harfbuzz = false \
  skia_use_system_icu = false \

[orto...@gmail.com]

Today the viewer ios app started working normally. I've then tried to use the lib file from viewer in the simple app  and it's crashing in __cxx_atomic_fetch_add.

I guess there is some missmatch between what xcode builds and what ninja builds, as viewer fully built with ninja has no issues. 

I don't think it's related to cflags/linker flags/preprocessor macros because I did bazel -> json -> cmake -> xcodeproj and I still can't build skia library without issues.

Any idea how to approach this?

[Jim Van Verth]

I took a look at it about a week and a half ago and I still couldn't duplicate the Viewer-side issues again, and your app kept crashing randomly on me. I haven't had a chance to track down the issue with the latter -- I might have some time later this week or early next week.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/556f0584-4cbd-49c1-8757-4cbfc4a66a03n%40googlegroups.com.

[orto...@gmail.com]

Hi, thanks for looking into it. I actually found the reason for the weird behaviour. In swapBuffersGraphite we need to first obtain the surface and do the drawing, then insert the recording and submit.

I made this silly mistake when implementing based on mac code. And about the crash inside libc, it was related to TRIVIAL_ABI flag. All is good now.

Being able to generate cmake project with  --ide=json --json-ide-script=../../gn/gn_to_cmake.py was of huge help to confirm that it was not a build system issue but code issue.

[Jim Van Verth]

Ah yes, that makes sense. Glad you figured it out.

To view this discussion on the web visit https://groups.google.com/d/msgid/skia-discuss/17f2a3f5-611b-4645-bc89-a9a183a304cen%40googlegroups.com.

[John Stiles]

It's intriguing that the TRIVIAL_ABI flag caused issues for you! On a Mac I would have assumed that a reasonably modern Clang was involved through the entire compile. Can you please elaborate on what you ran into?

[orto...@gmail.com]

I would not worry about it. I think I built skia with official_build=false, then use the static library in a xcode project. Adding SK_TRIVIAL_ABI=[[clang::trivial_abi]] to the xcode project of the app using the library fixed it.

When building with official_build=true there's no issue or need to add the flag.