M1 wheels for gRPC Python

[Philipp Moritz]

Dear gRPC Maintainers,

in my experience it is presently still very challenging to get gRPC for Python

working on Apple's M1 Platform ("Apple Silicon"). There have been various user

issues on github (see [1][2][3][4][5][6] for a partial collection of the issues)

regarding this, so I think many other members of the open source community

are facing very similar challenges.

This becomes increasingly more important as more open source projects are

building on top of gRPC, and each of these projects is inheriting all the

challenges around getting Python gRPC working on M1.

The issues are mainly about compiling gRPC and its dependencies (including

boringssl, zlib etc.) from the provided gRPC Python source distibutions. One possibility

to make this massively easier would be for Google/the gRPC team to provide binary

wheels for Python gRPC on M1 -- this would completely remove this set of friction

points for everybody. Currently there are lots of different workarounds that all

break in subtle and frustrating ways on different configurations.

I'm specifically asking on behalf of the Ray project (https://github.com/ray-project/ray),

currently we have to provide specialized instructions [7] to support M1 because there

are no gRPC M1 wheels (we basically have to fall back on conda's binary distribution,

which forces everybody to use conda).

You could improve the lives of everybody relying on Python gRPC on M1 greatly by

shipping M1 gRPC wheels on pypi, so people won't need to worry about getting a

working gRPC distribution.

Just wanted to get the discussion started and make sure this is on your radar and

check if there are plans for supporting it :)

I'd also like to take this opportunity to thank you all for making a lot of high quality

open source software available (e.g. gRPC, absl, bazel, K8s, TensorFlow) -- these are all

great to use, extremely well engineered and very useful enhancements of the

open source ecosystem!

All the best and thanks for your help,

Philipp.

[1] https://github.com/grpc/grpc/issues/25993

[2] https://github.com/grpc/grpc/issues/25082

[3] https://github.com/grpc/grpc/issues/28449

[4] https://github.com/grpc/grpc/issues/24846 (similar problems for Ruby)

[5] https://github.com/grpc/grpc/issues/26155

[6] https://github.com/grpc/grpc/issues/24390

[7] https://docs.ray.io/en/latest/installation.html?highlight=M1#m1-mac-apple-silicon-support

[Lidi Zheng]

Hi Philipp,

Thanks for writing up the good summary about the challenges we have around Apple Silicon.

I don't have a solution. But I can contribute following info: Apple is introducing "Universal Binary" to bundle binaries for arm64 and x86_64 into 1. Read more: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary.

M1 users can try to use `arch -x86_64 <command>` to use x86_64 binaries as Apple intended. I manually validated that this workaround is functional.

We do realize that this workaround is not native to normal Python workflow. We will bump up the priority to ship M1-native wheels.

[Philipp Moritz]

Hi Lidi,

Thanks for this tip, it is a great workaround and is making my life a bunch easier! I don't know if this is currently documented but it would be great to do that so other users can use it too :)

Also thanks for bumping the priority on M1-native wheels -- it will be great to have the Python ecosystem supporting M1 natively going forward and grpc is an important piece in the puzzle.

All the best,

Philipp.

[Pietro De Nicolao]

Hello Philipp,

I recently published the [grpcio-mac-arm-build](https://github.com/pietrodn/grpcio-mac-arm-build) repository, which contains a daily GitHub Actions job that cross-compiles the latest stable `grpcio` and `grpcio-tools` releases for Apple Silicon. The compiled wheels for Python 3.8, 3.9, 3.10, and 3.11 are available for download in the "Releases" page. Everything is open sourced and released with the same license of the `grpcio` library.

I hope Google will soon offer official builds.

Best regards,