iOS platform support for Metal ETA

[Robert Saget]

I've been tinkering around with the fork of ANGLE in the metalANGLE project (a fork of ANGLE: https://github.com/kakashidinho/metalangle). I think I have it working.

Any timeline or ETA on when ANGLE will supply platform support for iOS? The main page lists OSX as "in progress" and iOS as "planned".

Trying to figure out whether i should keep tinkering with metalANGLE or perhaps wait 6 months for ANGLE?

Thanks in advance!

[Thomas Burkhart]

Hi,

I would be highly interested in getting more information on your findings as I want to use metalangel too.

[Quyen Le]

Hi,

MetalANGLE and ANGLE mostly share the same Metal back-end's code atm. The only differences are:

1. MetalANGLE's code of other backends (Vulkan, etc) are quite out of date compare to current ANGLE's. If other back-ends are not main concerns, this could be ignored.
2. MetalANGLE can be built on iOS via Xcode project. ANGLE currently uses gn build scripts and it used to not support iOS in the past. There were some efforts to enable gn build scripts to compile for iOS simulator recently if I recall correctly.
3. MetalANGLE & ANGLE already have automated tests to verify the Metal back-end on macOS. There are no automated tests on iOS platform yet, hence on ANGLE readme file, it still lists iOS support as "planned". However, the Metal back-end's code is mostly the same between macOS & iOS.
4. MetalANGLE has Objective-C API to setup OpenGL context and integrate with iOS window's UI layers. This won't be upstreamed to ANGLE though. However, the two have the same lower level EGL API to setup GL window & context. MetalANGLE's Objective-C API is just a wrapper of this EGL API.

I believe if you want to try out ANGLE on iOS, you can use MetalANGLE atm. If 6 months later ANGLE officially supports iOS, you can switch to it if you want because on OpenGL API's point of view, there is no difference between the two.

The only problem is if you use MetalANGLE's Objective-C API to setup GL context, you will have to switch to use EGL API in the code instead. The better way could be using EGL API even if you use MetalANGLE, then later when switching to official ANGLE, you don't have to change any code.

[Quyen Le]

Oh, btw, there is also one more difference between MetalANGLE & ANGLE: MetalANGLE implements a custom extension to support importing Metal native texture to its GL context.
https://github.com/kakashidinho/metalangle/blob/master/extensions/EGL_MGL_texture_client_buffer.txt

This hasn't been upstreamed yet. I know official ANGLE currently has no need for this extension, but it could be useful for some users.

[Ken Russell]

We're actively collaborating with Apple on ANGLE's Metal backend. They're building on Quyen's tremendous work (thank you again Quyen!) and pushing for full OpenGL ES 3.0 conformance. This backend will support iOS.

There's going to be a quite large reconciliation of their work with Quyen's in the coming couple of months which I and others are going to help out with.

My recommendation would be to hold off a bit and wait until ANGLE's Metal backend has iOS support.

-Ken

--
You received this message because you are subscribed to the Google Groups "angleproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angleproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/02f9194f-841c-4620-8974-1ed71d4dc0dfn%40googlegroups.com.

[Robert Saget]

Thanks for the update Ken. And thanks Quyen!.

[John O'Connor]

It's been a few months but the github project does not indicate iOS metal support yet. Any movement on this? Is there a revised ETA?

[Ken Russell]

The authoritative repository for ANGLE is https://chromium.googlesource.com/angle/angle . Upstreaming of Apple's direct-to-Metal backend is actively underway in https://bugs.chromium.org/p/angleproject/issues/detail?id=5505 . We do not have an ETA for when this will be complete. This work is dovetailing with Apple's work on ANGLE's Metal backend in the WebKit repository, and both teams aim to converge on a common source base.

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/a0dd7356-5357-4f74-aa1d-19bd27b35852n%40googlegroups.com.

[Robert Saget]

Thanks for the update Ken.  I've been checking bi monthly for major updates. Apple hasn't pulled the plug on the OpenGL yet but I feel impending doom.

[Robert Saget]

Any update on ANGLE Metal backend support? I've got a a very light homegrown adapter I've been using but I need additional functionality. Debating whether (1) to burn some time updating my adapter, (2) take a stab at bringing in Quyen's adapter or (3) holding off for ANGLE to get support.

I've been trying to follow along progress with the following CRs, but its not real clear to me what the status is or how well things are going:
1. https://bugs.chromium.org/p/angleproject/issues/detail?id=5505 
2. https://bugs.chromium.org/p/angleproject/issues/detail?id=6394
3. https://bugs.chromium.org/p/angleproject/issues/list?sort=status%20priority&q=Renderer%3D%22Metal%22&can=1

Any update on progress/ETA would be appreciated so I can make a more informed decision. Thanks.

[Ken Russell]

Hi Robert,

This is being actively worked on by engineers from both Apple and Google. The top level bug is still:

https://bugs.chromium.org/p/angleproject/issues/detail?id=6394

You can see there is a fairly large bug tree associated with it. There's a lot of ongoing activity on bugs deeper in the tree.

I think perhaps the easiest way for you to keep tabs on the project is to make a saved query like:

Project(s): angleproject

Query: All issues

Renderer=Metal

Subscription options: Notify immediately

This is available by clicking on your logged-in name in the upper right corner of ANGLE's issue tracker, and selecting "Saved queries" from the drop-down.

We don't have a firm ETA for the completion of this work but are aiming to get it done as soon as possible - definitely before the end of Q4.

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/92fce475-b6f0-481c-9644-bf6d55d996dbn%40googlegroups.com.

[Robert Saget]

Hi Kenneth,

Thanks for the update and tips. Based on that info it seems like its in my best interests to wait a few more months since you think the Metal ANGLE work will be completed by then.

[Robert Saget]

Hi Kenneth,
I've been tracking the commits and CRs and it seems like platform support for Metal has been merged, to some extent. The official ANGLE webpage now lists iOS platform support as "in progress".

I noticed quite a few CRs are waiting related to Metal but they seem mostly like bugs and problems with non-essential functionality. 

https://bugs.chromium.org/p/angleproject/issues/list?q=Renderer%3DMetal

Is the ANGLE master branch in a state where I can start integrating ANGLE for iOS/Metal and expect it to work for basic use cases? Or is there a branch I can pull where something like that is available?

What is the current state of the iOS platform support?

TIA,

Robert

[Ken Russell]

Hi Robert,

Quite a lot of bugs have been fixed in the Metal backend in the past several months, and the majority of dEQP and WebGL conformance tests are passing on it. Apple is currently working on taking top-of-tree ANGLE back into WebKit:

https://bugs.webkit.org/show_bug.cgi?id=235746

and has been using the iOS backend in WebKit for some time. So yes, I think now's a fine time to look into using it.

You should set up your development processes so that you can easily roll forward ANGLE to a more recent version. You'll probably want to take another snapshot within a month because some critical customer bug fixes are still coming in, for example:

https://chromium-review.googlesource.com/c/angle/angle/+/3421527

https://chromium-review.googlesource.com/c/angle/angle/+/3472691

https://chromium-review.googlesource.com/c/angle/angle/+/3465937

Which build system will you be using? For example, raw Xcode, or CMake? We can make more investments in automatically generating CMake files from ANGLE's .gni files, but Xcode is out of reach; you'll have to update those project files by hand when you update ANGLE. For the record this is what's done in WebKit.

By the way, you can also see the changes Apple's made to their copy of ANGLE which haven't been upstreamed yet by checking this file:

https://github.com/WebKit/WebKit/blob/main/Source/ThirdParty/ANGLE/changes.diff

It's up to date immediately after ANGLE rolls into WebKit. For example, right now that file is a couple of months out of date.

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/a22e8d7d-c028-4ec3-abc1-ab316e728bb8n%40googlegroups.com.

[Robert Saget]

Hi Kenneth,
Thanks for the update.

To answer your question: The projects I'm looking to use ANGLE for use XCode. Developed in XCode on a Mac OS X machines. Compiled and deployed from a Mac OS X machines onto iOS devices.

I have other projects which use CMake or crosscompile to other architectures on Windows. I am considering ANGLE as a solution for those, especially with having to translate OpenGL to Vulkan. But its not the primary focus at the moment.

I'll take a look at integrating ANGLE with iOS platform support now since you indicated its in a stable enough state to use and past most conformance tests.

Are there any documents you recommend for integration besides the docs found here: https://github.com/google/angle/tree/main/doc 

[Ken Russell]

On Tue, Feb 22, 2022 at 3:35 PM Robert Saget <rsore...@gmail.com> wrote:

Hi Kenneth,

Thanks for the update.

To answer your question: The projects I'm looking to use ANGLE for use XCode. Developed in XCode on a Mac OS X machines. Compiled and deployed from a Mac OS X machines onto iOS devices.

I have other projects which use CMake or crosscompile to other architectures on Windows. I am considering ANGLE as a solution for those, especially with having to translate OpenGL to Vulkan. But its not the primary focus at the moment.

I'll take a look at integrating ANGLE with iOS platform support now since you indicated its in a stable enough state to use and past most conformance tests.

Are there any documents you recommend for integration besides the docs found here: https://github.com/google/angle/tree/main/doc 

I'd suggest you look at how WebKit creates its EGL context for WebGL rendering via ANGLE:

https://github.com/WebKit/WebKit/blob/main/Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.mm

Look at the EGL_CreateContext call and surrounding code, including how it binds pbuffers to IOSurfaces for on-screen display via CALayer. This is similar to how Chromium uses ANGLE, but is more self-contained and easy for a newcomer to the code to understand.

-Ken

 

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/e7268633-e277-4ae0-9a4f-00e2f69d3c47n%40googlegroups.com.

[Linfeng Li]

HI Kenneth. I was able to build the Mac version of ANGLE successfully using GN and Ninja. However the GN command just hangs forever if I set target_os to ios.

is_debug = true
target_cpu = "x64"
target_environment = "simulator"
target_os = "ios"

After gn args ..., it just hangs with output

Waiting for editor on "/Users/xxx/args.gn"...
Generating files...

Another thing I find it weird on Mac was eglChooseConfig is crashing. It seems that the arguments in this function have wrong sizes. EGLDisplay is void *, and should be 8bytes, but the dpy in the bottom left window suggests it is larger than that. I was able to work around it by changing it to direct call to EGL_ChooseConfig, and the app is working. (I used MetalANGLE + EGL in the past, so almost no change needed to be made other than linking to libEGL.dylib and libGLESv2.dylib instead of MetalANGLE.xcframework)

[Ken Russell]

Hi Linfeng,

On Fri, Feb 25, 2022 at 7:32 PM Linfeng Li <crye...@gmail.com> wrote:

HI Kenneth. I was able to build the Mac version of ANGLE successfully using GN and Ninja. However the GN command just hangs forever if I set target_os to ios.

is_debug = true
target_cpu = "x64"
target_environment = "simulator"
target_os = "ios"

After gn args ..., it just hangs with output

Waiting for editor on "/Users/xxx/args.gn"...
Generating files...

Yuly Novikov (CC'd) has been working on getting ANGLE's Metal backend compiling and running on Chromium's iOS Simulator bots. Yuly, could you point to any bugs in progress?

Linfeng - at the moment ANGLE's iOS Simulator build is primarily known to work inside Xcode; the WebKit project uses that build configuration.

 

Another thing I find it weird on Mac was eglChooseConfig is crashing. It seems that the arguments in this function have wrong sizes. EGLDisplay is void *, and should be 8bytes, but the dpy in the bottom left window suggests it is larger than that. I was able to work around it by changing it to direct call to EGL_ChooseConfig, and the app is working. (I used MetalANGLE + EGL in the past, so almost no change needed to be made other than linking to libEGL.dylib and libGLESv2.dylib instead of MetalANGLE.xcframework)

I'm not sure why this is happening - perhaps you're accidentally pulling in a system EGL header rather than ANGLE's headers. It's necessary for your application to use ANGLE's headers exclusively.

WebKit's ANGLE backend for WebGL uses the ANGLE-internal EGL_ChooseConfig and similar entry points specifically to avoid any such confusion. Look in https://github.com/WebKit/WebKit under Source/WebCore/platform/graphics/cocoa/GraphicsContextGLCocoa.mm and Source/WebCore/platform/graphics/angle/ .

-Ken

 

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/896d982f-f18d-4d3c-97ee-c81fead2c2a9n%40googlegroups.com.

[Yuly Novikov]

I've never built for iOS in Debug.

We have this bot, which doesn't work yet, but I was able to build locally with similar GN args

https://luci-milo.appspot.com/p/chromium/builders/ci/ios-angle-builder

I'd expect something like this to work, but I don't have a Mac to try this myself:

is_component_build = false
is_debug = false
symbol_level = 1

target_cpu = "x64"
target_environment = "simulator"
target_os = "ios"

Tracking bug for making this bot work is anglebug.com/5417.

You could also try the args James used in https://bugs.chromium.org/p/angleproject/issues/detail?id=5417#c11.

Note that iOS build is not supported in standalone ANGLE, but only for ANGLE built from the Chromium repo.

As for the progress on that bug, it is currently waiting for https://chromium-review.googlesource.com/c/chromium/src/+/3035587 to land.

We have 2 options to proceed with it:

1. Fix anglebug.com/6937, which is blocked on b/216733833

2. Split https://chromium-review.googlesource.com/c/angle/angle/+/3399045 and land this one line https://chromium-review.googlesource.com/c/angle/angle/+/3399045/5/src/tests/test_utils/runner/TestSuite.cpp#1370

[Ken Russell]

Thanks Yuly for your input and the background.

I'd like to advocate for removing the dependency on the Leak Sanitizer (LSAN) in whatever way possible. We would get good utility from having the iOS Simulator bot up and running tests even if LSAN isn't working.

-Ken

[Linfeng Li]

I just discovered I could append -v to gn to get a verbose output so I found out it hangs because it was trying to find code sign identity again and again. I waited a little bit more and the gn args command finishes.

I managed to get it working now after some tweaks to the code.

1. I need to make some changes to src/compiler/translator/tree_ops/vulkan/EarlyFragmentTestsOptimization.h.  although I set enable_vulkan to false it still tries to compiles some of the source files in src/compiler/translator/tree_ops/vulkan 

2. src/libANGLE/renderer/metal/BUILD.gn add linking to "IOSurface.framework" and "QuartzCore.framework"

3. Fix src/common/system_utils_posix.cpp, the existing code App/libGLES.framework/libGLES.framework is used for dlopen, where App/libGLES.framework/libGLES should be used instead

.

I still cannot get the catalyst version of ANGLE to build

It errors:

ninja: Entering directory `out/iOS'
ninja: error: '../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/Agents/XCTRunner.app/XCTRunner', needed by 'obj/build/config/ios/XCTRunner', missing and no known rule to make it

Looks like test related? Can I maybe just disable building tests?

> I'm not sure why this is happening - perhaps you're accidentally pulling in a system EGL header rather than ANGLE's headers. It's necessary for your application to use ANGLE's headers exclusively.

Mac does not have EGL so there is no system header I think. I was not able to reproduce it on iOS, which is weirder because I actually expected the iOS version to have the same issue.

[Jamie Madill]

Additionally to Yuly's two options, a third option is to resolve http://crbug.com/1293017

If I were driving this work that's what I would pursue.

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/11b8ec2b-1cad-48c0-afe6-048424c6c52dn%40googlegroups.com.

[Ken Russell]

On Tue, Mar 1, 2022 at 2:40 AM Linfeng Li <crye...@gmail.com> wrote:

I just discovered I could append -v to gn to get a verbose output so I found out it hangs because it was trying to find code sign identity again and again. I waited a little bit more and the gn args command finishes.

OK. If there's something that should be changed in gn please let us know.

 

I managed to get it working now after some tweaks to the code.

Thanks Linfeng for your updates, that's great news!

 

1. I need to make some changes to src/compiler/translator/tree_ops/vulkan/EarlyFragmentTestsOptimization.h.  although I set enable_vulkan to false it still tries to compiles some of the source files in src/compiler/translator/tree_ops/vulkan 

2. src/libANGLE/renderer/metal/BUILD.gn add linking to "IOSurface.framework" and "QuartzCore.framework"

3. Fix src/common/system_utils_posix.cpp, the existing code App/libGLES.framework/libGLES.framework is used for dlopen, where App/libGLES.framework/libGLES should be used instead

Would you mind putting up a CL containing these - ideally cleaned up, and definitely tested locally? For example if we're compiling Vulkan files we shouldn't be, let's fix the GN files.

https://chromium.googlesource.com/angle/angle/+/refs/heads/main/doc/ContributingCode.md

 

I still cannot get the catalyst version of ANGLE to build

It errors:

ninja: Entering directory `out/iOS'
ninja: error: '../../../../../../../../../Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Xcode/Agents/XCTRunner.app/XCTRunner', needed by 'obj/build/config/ios/XCTRunner', missing and no known rule to make it

Looks like test related? Can I maybe just disable building tests?

How are you doing this build? I don't have any experience with Mac Catalyst.

 

> I'm not sure why this is happening - perhaps you're accidentally pulling in a system EGL header rather than ANGLE's headers. It's necessary for your application to use ANGLE's headers exclusively.

Mac does not have EGL so there is no system header I think. I was not able to reproduce it on iOS, which is weirder because I actually expected the iOS version to have the same issue.

OK, sorry, I'm not sure what is going on here.

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/11b8ec2b-1cad-48c0-afe6-048424c6c52dn%40googlegroups.com.

[Linfeng Li]

yeah I'll try to upstream the changes in groups. it is relatively straightforward.

> How are you doing this build? I don't have any experience with Mac Catalyst.

Good news is that I could get the catalyst build to work too. Need to change some code in gni files. It can be built by setting target_os = "ios" and target_environment = "catalyst"

> For example if we're compiling Vulkan files we shouldn't be, let's fix the GN files.

I think it is introduced here, not sure the reason tho based on the comment

      if (angle_enable_vulkan || use_fuzzing_engine || angle_enable_metal) {
        _needs_glsl_base = true
        _needs_glsl_and_vulkan_base = true
        _uses_spirv = true

        # This translator is needed by metal backend also.
        sources += angle_translator_lib_vulkan_sources
      }

[Ken Russell]

On Tue, Mar 1, 2022 at 10:00 PM Linfeng Li <crye...@gmail.com> wrote:

yeah I'll try to upstream the changes in groups. it is relatively straightforward.

Great, thank you.

 

> How are you doing this build? I don't have any experience with Mac Catalyst.

Good news is that I could get the catalyst build to work too. Need to change some code in gni files. It can be built by setting target_os = "ios" and target_environment = "catalyst"

> For example if we're compiling Vulkan files we shouldn't be, let's fix the GN files.

I think it is introduced here, not sure the reason tho based on the comment

      if (angle_enable_vulkan || use_fuzzing_engine || angle_enable_metal) {
        _needs_glsl_base = true
        _needs_glsl_and_vulkan_base = true
        _uses_spirv = true

        # This translator is needed by metal backend also.
        sources += angle_translator_lib_vulkan_sources
      }

I think that the direct-to-Metal backend used some shader translator passes from the Vulkan backend without properly moving them out of src/compiler/translator/tree_ops/vulkan/ into a higher-level directory. If you have time to look into this more deeply we'd certainly appreciate it, since we aren't seeing this issue when building the Metal backend for macOS.

Thanks,

-Ken

 

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/35de886b-2d94-4d35-bcd1-5c478414da00n%40googlegroups.com.