How we cross-compiled libwebrtc from GNU/Linux to Mac OS X

[David Fifield]

Some of us are working on a standalone program using the WebRTC native
code package (no browser or JS). This program will eventually become
part of Tor Browser and enable Tor access over WebRTC. Integrating with
the native code package has been a challenge, particularly given the
constraints of Tor Browser's Gitian-based reproducible build system
(using XCode is not an option for us). So far, we have things working
for GNU/Linux and Mac, and haven't tried Windows yet. GNU/Linux was not
too difficult. This is a description of what tweaks we've had to apply
to get the Mac build working. I hope that someone can show us a better
way to do these things, or the upstream changes to make them easier.

This is the main ticket tracking our progress:
https://bugs.torproject.org/19001
We use the WebRTC native code package through this Go wrapper:
https://github.com/keroserene/go-webrtc

Here is our complete build script. It's run in a Debian wheezy VM.
https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/tree/gitian/descriptors/mac/gitian-webrtc.yml?h=snowflake&id=f811240035c3ac01818f44eb606d08c532cb1e61

The biggest aspect of our changes is this patch applied directly to the
webrtc source:
https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/tree/gitian/patches/webrtc-mac.patch?h=snowflake&id=f811240035c3ac01818f44eb606d08c532cb1e61
[PATCH 01/13] "find -E" doesn't work with GNU find.
[PATCH 02/13] Our cross-ld for Mac doesn't support -z and other linker flags.
[PATCH 03/13] Put "-U__APPLE__" in cflags, not just xcode_settings.
[PATCH 04/13] Include <sys/socket.h> for MacOSX10.7.sdk compatibility.
[PATCH 05/13] Have gyp emit .m and .mm targets even when not running on a Mac.
[PATCH 06/13] Put "-framework Foundation" in ldflags, not just xcode_settings.
[PATCH 07/13] Provide definitions of NS_ENUM and NS_OPTIONS.
[PATCH 08/13] Don't use objc++ lightweight generics.
[PATCH 09/13] Put "-fobjc-arc" in cflags, not just xcode_settings.
[PATCH 10/13] Use objectAtIndex.
[PATCH 11/13] Put "-fobjc-arc" in cflags, not just xcode_settings.
[PATCH 12/13] Use objectAtIndex.
[PATCH 13/13] Provide definitions of NS_ENUM and NS_OPTIONS.
Some of the changes have to do with the build assuming it's happening on
a Mac (01, 02, 05). Some copy settings from xcode_settings to the
ordinary cflags and ldflags (03, 06, 09, 11). The remaining changes just
have to do with the older SDK and compiler we're currently using
(04, 07, 08, 10, 12, 13).

The build/mac/find_sdk.py script refuses to run if not on a Mac. We
hacked it to just emit our preconfigured $SDKROOT:
echo "print(\"$SDKROOT\")" > build/mac/find_sdk.py

We're building against MacOSX10.7.sdk (the build seems to assume
>=10.8), so we had to provide definitions of NS_ASSUME_NONNULL_BEGIN and
NS_DESIGNATED_INITIALIZER in order to get a few files to compile.

I don't know why, but gyp_webrtc.py leaves literal '$(SDKROOT)' in some
ninja files, which causes ninja to choke when run. We rewrite all of
those:
find out/Release -name '*.ninja' -print0 | xargs -0 sed -i -e 's#\$(SDKROOT)#'"$SDKROOT"'#g'

Some of the ninja files under obj.host seem to erroneously use LDFLAGS
instead of LDFLAGS_host, so we have to rewrite them:
find out/Release/obj.host -name '*.ninja' -print0 | xargs -0 sed -i -e '/^ldflags = .*[^$]$/ cldflags ='"$LDFLAGS_host" -e '/^ldflags = .*\$$/,/[^$]$/ cldflags ='"$LDFLAGS_host"

We didn't find any preexisting target to build a "libwebrtc.a". Instead,
we build the default ninja target and then grab all the .o files:
$AR crs libwebrtc-magic.a $(find . -name '*.o' -not -name '*.main.o' -not -name 'dump_syms_regtest.o')
And .h files:
INCLUDE_DIR="$PWD/webrtc/include"
(cd $HOME/build/webrtc/src && for h in $(find talk/ webrtc/ -type f -name '*.h'); do
mkdir -p "$INCLUDE_DIR/$(dirname $h)"
cp -f "$h" "$INCLUDE_DIR/$h"
done)

We'd appreciate any advice on how to do this better, especially as
regards not having to maintain a patch against the webrtc code.

[David Fifield]

Last year I wrote to this list with a summary of how we were able to
make libwebrtc cross-compile from Debian GNU/Linux for macOS.
https://groups.google.com/forum/#!msg/discuss-webrtc/ca_R9ocEHus/Miu30AXPAwAJ

That message was using GYP. Recently we updated the patches to work with
GN. The changes actually got a bit simpler. As before, suggestions about
how better to accomplish this, or ideas about making this
cross-compilation a supported configuration, are welcome.

Here's a summary of the changes on our issue tracker:
https://bugs.torproject.org/19001#comment:32
Here is the build script. The main trick here is to rewrite find_sdk.py
and sdk_info.py to hardcode our SDK information so as not to require
Xcode command-line utilities.
https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/tree/gitian/descriptors/mac/gitian-webrtc.yml?h=snowflake&id=2edd40a4da8abd4581846d403a0db994869e2cc7
The other part is this patch applied to the webrtc source code. The
first three patches have to do with cross-compiling; the other five have
to do with the 10.7 macOS SDK we use.
https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/tree/gitian/patches/webrtc-mac.patch?h=snowflake&id=2edd40a4da8abd4581846d403a0db994869e2cc7
[PATCH 1/8] Disable assertions that prevent cross-compiling for mac.
[PATCH 2/8] Hardcode cross-compiling flags in config/mac/BUILD.gn.
[PATCH 3/8] Port build/config/mac/plist_util.py to biplist.
[PATCH 4/8] Hack workaround to disable dispatch_queue_get_label.
[PATCH 5/8] Include <sys/socket.h> in macifaddrs_converter.cc.
[PATCH 6/8] Provide definitions of NS_ENUM and NS_OPTIONS.
[PATCH 7/8] Replace firstObject with objectAtIndex:0.
[PATCH 8/8] Disable the desktop_capture module.