Chromium 64-bit on Mac OS

[catalinb]

Hello chromium-dev,

The 64-bit mac build is mostly error free. Currently, the 64-bit version can be built and run without much trouble from xcode with the appropriate target_arch and ARCHS flags set. There are a couple of issues I'd like to point out and discuss. I am mainly interested in what is required for having a separate buildbot for the 64-bit version and how should gyp or the gyp files be improved to support it.

Regarding the universal build, It is my understanding that a fat binary is preferred over two separate 32-bit and 64-bit versions. One of the reasons, as Stuart Morgan pointed out, is that a thin 64-bit build would break compatibility with some of the current available plugins. That is true, but shouldn't the plugins be packaged as fat binaries instead, thus supporting both versions of chromium? What are the other reasons for wanting universal binaries?

Getting gyp to support universal builds will be a bit tricky, since the ninja generator parses the xcode setting and actually uses a hardcoded -arch i386 flag. On the other hand, xcode can have multiple architectures specified through the ARCHS flags but won't use the target_arch variable which used for including/excluding sources from the build. A reasonable first step would be to just get gyp to generate projects for 32-bit or 64-bit targets depending on the target_arch variable. Would that be acceptable?

About supporting NaCl, the x64 NaCl sandbox is not currently available on 64-bit. I've been using 'disable_nacl=1' to remove native client from the build. Is NaCl support mandatory for the time being?

http://code.google.com/p/nativeclient/issues/detail?id=3087

Related issue: http://code.google.com/p/chromium/issues/detail?id=18323

Regards,

Catalin Badea

[Stuart Morgan]

2012/11/7 catalinb <ba...@adobe.com>

That is true, but shouldn't the plugins be packaged as fat binaries instead, thus supporting both versions of chromium?

We don't control those plugins; we're talking about NPAPI plugins distributed by third-party vendors. Both Apple's WebKit framework (and by extension Safari and many other Mac browsers) and Gecko (and by extension Firefox) support 32-bit plugins in their 64-bit builds on OS X, so from the perspective of many plugin vendors there has never been a need to ship a 64-bit version on OS X. Whatever anyone might think about whether NPAPI plugins *should* be fat, the reality is that a number of very popular plugins *aren't*.

In addition, QuickTime is bundled with the OS, and is 32-bit only. It seems extremely unlikely that Apple has plans to add a 64-bit version of that plugin retroactively all the way back to 10.6, given than other QuickTime plugin changes weren't retroactive (e.g., 10.5 never got a version of the plugin that supported the modern NPAPI drawing and event models).

 

What are the other reasons for wanting universal binaries?

Why do we need other reasons?

-Stuart

[Mike Reed]

Across a (admittedly finite) swath of web pages, Skia draws roughly 10% faster in 64bit (some extreme sites: gmail:20% faster, mlb:1.8x faster). Is there a path to ship a 64bit version of chrome for Mac (either fat or otherwise)?

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Mark Mentovai]

I’m looking at this now.

[Lambros Lambrou]

On Wed, Nov 7, 2012 at 6:45 AM, catalinb <ba...@adobe.com> wrote:

Hello chromium-dev,

The 64-bit mac build is mostly error free. Currently, the 64-bit version can be built and run without much trouble from xcode with the appropriate target_arch and ARCHS flags set. There are a couple of issues I'd like to point out and discuss. I am mainly interested in what is required for having a separate buildbot for the 64-bit version and how should gyp or the gyp files be improved to support it.

Regarding the universal build, It is my understanding that a fat binary is preferred over two separate 32-bit and 64-bit versions. One of the reasons, as Stuart Morgan pointed out, is that a thin 64-bit build would break compatibility with some of the current available plugins. That is true, but shouldn't the plugins be packaged as fat binaries instead, thus supporting both versions of chromium? What are the other reasons for wanting universal binaries?

Having base/ (and, to a lesser extent, crypto/) built as universal binaries would be very useful for Chromoting on Mac. We build a System Preferences plugin, which needs to be a universal binary. So, currently, we can't link with any Chrome libraries (so that any code we do reuse has to be either copy-pasted, or added directly as source/header files to our project). This is a hacky solution that is likely to hamper future work on this pref-pane (for example, localization may be difficult).

Lambros

[Marc Widmer]

I'm actually no developer, but I'd like to have a 64bit version of Chrome(ium) for mac os x due to my needs to use chrome and new java 7 and I don't want to do it on a win oder linux pc. Does anyone can provide me witha a 64bit chrome(ium) for download? 

[Rick Mills]

What I'd like to know is why we even need a 32bit version anymore? Macs havent shipped with 32-bit exclusive processors in a long time now. OS X has been 64bit for over 5 years now!

Chromium/Chrome developers: Get with the damn program please. There's no excuse for not pushing for a 64bit release in this day and age.

[Catalin Badea]

I can confirm that the 64-bit Chromium build runs the java plugin (v7 update 10). Should I provide you with a patch that enables the 64-bit Chromium build?

On Jan 9, 2013, at 1:05 PM, Marc Widmer <marc....@gmail.com> wrote:

I'm actually no developer, but I'd like to have a 64bit version of Chrome(ium) for mac os x due to my needs to use chrome and new java 7 and I don't want to do it on a win oder linux pc. Does anyone can provide me witha a 64bit chrome(ium) for download? 

[Stuart Morgan]

2013/1/9 Rick Mills <rick.s...@gmail.com>:

> What I'd like to know is why we even need a 32bit version anymore?

That was already addressed in this thread.

> Chromium/Chrome developers: Get with the damn program please.

This list is for discussing the actual development of Chromium, not
complaining about feature/bug {X} not being implemented/fixed yet.
There's chromium-discuss or the forums if you really need to do that.
Wasting the time of everyone working on Chromium by complaining here
doesn't get things done faster; it does the opposite.

-Stuart

[Paweł Hajdan, Jr.]

Patches are welcome if you have them! Please see http://dev.chromium.org/developers/contributing-code for the process. The tracking bug is https://code.google.com/p/chromium/issues/detail?id=18323 .

Mark Mentovai (mark@) who is the owner of the bug is also CC-ed here, so feel free to coordinate with him.

I don't want your patch to get stuck by the way, so if that doesn't get any progress feel free to ping me.

Paweł

[Nico Weber]

On Wed, Jan 9, 2013 at 10:03 AM, Paweł Hajdan, Jr.
<phajd...@chromium.org> wrote:
> Patches are welcome if you have them! Please see
> http://dev.chromium.org/developers/contributing-code for the process. The
> tracking bug is https://code.google.com/p/chromium/issues/detail?id=18323 .
>
> Mark Mentovai (mark@) who is the owner of the bug is also CC-ed here, so
> feel free to coordinate with him.
>
> I don't want your patch to get stuck by the way, so if that doesn't get any
> progress feel free to ping me.

As far as I know, the tricky part for 64bit isn't the patches as much
as the build infra and deployment work that needs to happen for this
behind the scenes. Pawel, please leave approval of 64bit stuff up to
Mark.

[catalinb]

On Wednesday, January 9, 2013 8:03:44 PM UTC+2, Paweł Hajdan, Jr. wrote:

Patches are welcome if you have them! Please see http://dev.chromium.org/developers/contributing-code for the process. The tracking bug is https://code.google.com/p/chromium/issues/detail?id=18323 .

Mark Mentovai (mark@) who is the owner of the bug is also CC-ed here, so feel free to coordinate with him.

I don't want your patch to get stuck by the way, so if that doesn't get any progress feel free to ping me.

Paweł

Thank you!

I think I've made some progress with the universal build. I managed to build a fat Chromium binary using Xcode and I'll begin submitting the required patches for code review. While the generated xcode projects work well and produce a universal binary as intended, I'm not sure that my approach is entirely clean. As such, I'd like to describe the changes that I've made and, maybe, you could give me some suggestions.

1. Use 'mac_universal' as a target_arch condition. Generating the .xcodeproj can be done with GYP_DEFINES='target_arch="mac_universal" build/gyp_chromium;

2. Enable xcode_settings: ARCHS: $(ARCHS_STANDARD_32_64_BIT) from build/common.gypi. This condition makes XCode build each target twice and obtain a universal binary using lipo.

3. Replace gyp 'defines' entries enclosed in target_arch=<ia32/x64> conditions with GCC_PREPROCESSOR_DEFINITIONS[arch=<i386/x86_64>]. This works surprisingly well provided that a current bug in gyp is fixed first(https://codereview.chromium.org/12094059/).

4. Add 'mac_universal' as a valid match for conditions that match both i386 and x86_64. 'target_arch=="ia32" or target_arch=="x64" or target_arch=="mac_universal"' looks rather ugly - maybe gyp can be improved to perform a smarter matching?

With these changes, targets with C/C++/ObjC code should build without any errors. One limitation from which XCode currently suffers, afaik, is that you cannot specify different sets of sources for each target_arch. This means that each source is compiled twice regardless if it is actually used on architecture you're building. The only solution I've found to this is using preprocessor conditions for excluding/including code.

The ugly part: third_party targets which use .asm sources.

Third party dependencies such as libvpx, ffmpeg and libjpeg_turbo use hand compiled .asm sources.

5. Modify the yasm_compile build rule to build each .asm source twice and merge the two objects into one using lipo.

6. The new build rule uses yasm_flags_ia32 and yasm_flags_x64. Therefore, break yasm_flags entries into specialized entries.

Note: XCode links correctly against fat objects.

The downside: you don't have two separate sets of sources and the effects are more visible here: there are a lot of asm sources that should be built for only one architecture. I used preprocessor conditions to fix this, but I don't believe that this is the right way.

7. The assembled objects are parsed/extracted with some custom rules for generating unpacked libs or lists of integer offsets. These rules need to be modified to work with fat objects. These include gen_asm_offets and unpack_lib.

8. Build fat Chromium for greater glory.

I have not tested the ninja build, but I believe that the ninja generator should parse ARCHS and do something similar to XCode.

The IPCs between the main process in 64-bit mode and a helper process in 32-bit mode don't work, which was expected.

Any suggestion is greatly appreciated.

Regards,

Cătălin Badea

 

[Mark Mentovai]

Ultimately, I would like to use 'mac' (not 'mac_universal') as the target_arch and host_arch for this, and switch to just using 'mac' as the default target_arch and host_arch for all Mac builds (even single-arch), to prevent it from diverging from the universal build.

[Ryan Sleevi]

The design of GYP was trying to find a unified syntax that works for
all generators - including MSVS and Make. The existing target_arch
meets this requirement, but the proposed change only works for the
XCode generator. This is also why, for example, per-configuration
sources are not permitted, even though they're supported on all
generators EXCEPT XCode, and even though they would be quite useful
for multi-arch.

I think the GYP issue you refer to is more complex than you may
realize - and it would be useful to find a solution that is also
viable for Windows builds, given the x64 build work being led by
jschuh@.

[catalinb]

I did not meant breaking configurations for the current builds, just adding

additional options for the mac build.

Regarding the use of per-architecture preprocessor definitions, I noticed

they're needed for a single target (libsrtp), therefore, I'll use a workaround.

The use of the 'ARCHS' options is an easy way for getting XCode to build

universal binaries and Ninja can be extended to generate similar build

rules, as well. Since fat binaries are only needed on MacOS, wouldn't the

use of 'ARCHS' be justified?

Does the windows build need a similar 32-bit/64-bit multi-arch solution?

On Thursday, January 31, 2013 9:26:17 PM UTC+2, Mark Mentovai wrote:

Ultimately, I would like to use 'mac' (not 'mac_universal') as the target_arch and host_arch for this, and switch to just using 'mac' as the default target_arch and host_arch for all Mac builds (even single-arch), to prevent it from diverging from the universal build.

That seems sensible.

[Alexander Gilburg]

I can confirm that Chromium (64-bit) indeed builds with the "-Dtarget_arch=x64" flag passed to gyp error-free on OS X using the default codebase with no modifications applied to it (directly from git). The binary exceeds 500 MB, so it will be problematic to get officially supported builds. Not even mentioning hybrid 32/64-bit builds, the size will be tremendous. As Badea and Morgan have mentioned, 64-bit only will certainly break compatibility with some, but not all existing plugins.

The NaCl issue was resolved in the latest git-revisions, since the "Bullet Physics NaCl Test" extension (https://chrome.google.com/webstore/detail/bullet-physics-nacl-test/pgehkhceingafmkkmbeoempaablkkeal) testing the Native Client runs well on OS X.

If, somehow, the file-size of the binary gets reduced, we can start to consider of releasing 64-bit versions of Chromium for OS X. Also, Chromium 64-bit build just crashed on me while I was writing this (thanks to the devs who've implemented the restoration feature for not losing this text), but it most likely has something to do with the most recent git-revision.

I am also posting a screenshot of Chromium 64-bit running side-by-side with Google Chrome:

Alexander Gilburg,

03.01.14

[Torne (Richard Coles)]

On 3 January 2014 13:00, Alexander Gilburg <alexande...@gmail.com> wrote:

I can confirm that Chromium (64-bit) indeed builds with the "-Dtarget_arch=x64" flag passed to gyp error-free on OS X using the default codebase with no modifications applied to it (directly from git). The binary exceeds 500 MB, so it will be problematic to get officially supported builds. Not even mentioning hybrid 32/64-bit builds, the size will be tremendous. As Badea and Morgan have mentioned, 64-bit only will certainly break compatibility with some, but not all existing plugins.

The binary being 500MB suggests it has some debugging info/symbols built into it... default builds of chromium aren't configured for building official release builds without passing extra gyp options. So, this probably isn't an actual problem.

 

The NaCl issue was resolved in the latest git-revisions, since the "Bullet Physics NaCl Test" extension (https://chrome.google.com/webstore/detail/bullet-physics-nacl-test/pgehkhceingafmkkmbeoempaablkkeal/related) testing the Native Client runs well on OS X.

If, somehow, the file-size of the binary gets reduced, we can start to consider of releasing 64-bit versions of Chromium for OS X. Also, Chromium 64-bit build just crashed on me while I was writing this (thanks to the devs who've implemented the restoration feature for not losing this text), but it most likely has something to do with the most recent git-revision.

I am also posting a screenshot of Chromium 64-bit running side-by-side with Google Chrome:

Alexander Gilburg

--

--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Alexander Gilburg]

Thanks for pointing the thing with the debugging symbols out, I will now try and do a fastbuild with dbg disabled and see what the actual size will be...

Regards,

Alexander Gilburg

[Torne (Richard Coles)]

There are more flags than that which impact binary size (e.g. see "logging_like_official_build" and various others), but making sure you're omitting all debug info and stripping all symbols is the main one, yes :)

--

[Mark Mentovai]

All of these problems have been solved or are not problems at all. In fact, we’ve already done a 64-bit canary release as a test and are planning the roadmap for a larger-scale release.

https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/LBHN-SWwI7I

[Alexander Gilburg]

So, is there any ETA for "official" 64-bit Chromium releases?