Using system HarfBuzz on non-official builds.

[Emil A Eklund]

[was Linux GN build won't link, harfbuzz problems?]

For linux we use the system version of HarBuzz except for official
builds where we use the version embedded in third_party. I can
understand why we'd like to use the system version but having
developer and official builds use different versions of a core library
seem like a bad idea to me. Especially since we often make changes to
harfbuzz and depend on new functionality added to it.

Not only that, having developers build and tests against a different
version than our end users seems really undesirable.

Do we have any plans to fix this and use the embedded version for all
builds, instead of just the official ones?

> Please start a separate thread for the general problem of using a pinned
> version of harfbuzz vs. the system-provided one, but generally speaking
> this is a problem we have with a bunch of Linux libraries, I think, and I
> think we're actually doing what the distros "want" us to be doing. Of
> course, doing so does land us in DLL hell ...

Except we don't do it for official builds... I'd argue that we should
either use the system version everywhere or nowhere. Of course if we
do decide to use the system version then our iteration speed when it
comes to text rendering is limited to the release schedule of the
linux releases we want to support.

--
Emil

[Brett Wilson]

I think defaulting to the built-in one is a good idea. I'm not sure why it was hooked up this way originally.

Brett

[Dirk Pranke]

I hazard a guess that we use the system-provided one for chromium builds because some distros (which build chromium, not chrome) might prefer it that way.

I concur that that should not be the default mode of operation for us; we should try to match the official builds as much as is practical.

-- Dirk

[Jochen Eisinger]

We bundled harfbuzz, because the system version on the linux distro we used internally wasn't recent enough. With a recent enough distro, it should be possible to use the system version.

In fact, on ChromeOS, where we have the system under control, we want to use the system version instead of the bundled version, also for official builds.

best

-jochen

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Paweł Hajdan, Jr.]

Re-posting my answer from original thread https://groups.google.com/a/chromium.org/d/msg/chromium-dev/LcBIzQjfOXI/NnMIP4d5wjIJ :

Just to clear a possible confusion: distros have nothing to do with what Google Chrome does. It can bundle the universe.

Distro packaging guidelines apply to chromium builds, but we do have a solution for that as well (build/linux/unbundle) which is transparent to Chrome developers and works well for distros.

Bottom line: don't worry about distros here. Do the right thing for the project.

Paweł

[Emil A Eklund]

On Tue, Feb 24, 2015 at 1:37 AM, Jochen Eisinger <joc...@chromium.org> wrote:
> We bundled harfbuzz, because the system version on the linux distro we used
> internally wasn't recent enough.

Since we switched blink to harfbuzz for shaping on all platforms we've
had to make a number of changes that require both a blink and a
harfbuzz change. Recent work on the coretext shaper and work last year
on the windows implementation to unblock the switch to harfbuzz on mac
and windows respectively demonstrates this pretty clearly. If we had
to depend on the system version of harfbuzz on those platforms the
switch to harfbuzz would have been problematic at best.

From what I can tell the only reason we use the system version at all
is because pango started pulling in harfbuzz and, at the time, the
version provided by pango was sufficient to support chrome at the
time.

> With a recent enough distro, it should be
> possible to use the system version.

It is for the current version of chrome. We need a newer version of it
for some text improvements we're working on. Despite the fact that the
never version being in third_party and is being used for official
builds we cannot use it as linux distributions bundle a much older
version. On my (current and supported) ubuntu system the version of
harfbuzz is almost a year old.

Depending on packages that are that out of date really isn't practical
and effectively means that we cannot make any improvements to text
shaping or text shaping performance without waiting for a full release
cycle of every single linux distribution we support.

> In fact, on ChromeOS, where we have the system under control, we want to use
> the system version instead of the bundled version, also for official builds.

That is perfectly fine as long as there is a way to update it.

What would it entail to have all linux builds go down the
official-build path and _never_ use the system version of harfbuzz?

[Jochen Eisinger]

from my point of view, always using the bundled version (modulo whatever needs to be done for ChromeOS) is fine.

Whoever wants to use the system harfbuzz library should speak up so we can figure out a solution.

best

-jochen

[Paweł Hajdan, Jr.]

On Tue, Feb 24, 2015 at 10:35 PM, Jochen Eisinger <joc...@chromium.org> wrote:

from my point of view, always using the bundled version (modulo whatever needs to be done for ChromeOS) is fine.

Whoever wants to use the system harfbuzz library should speak up so we can figure out a solution.

I wouldn't expect anyone to speak up. Most distro folks for example don't have enough time to handle blink-dev traffic to watch for threads like this.

Don't worry though. I think it's fine for the project to move to bundled harfbuzz. I'm excited about more improvements to text shaping this would enable, and changes to harfbuzz eventually benefit all users of that library. As mentioned before, we have existing infrastructure for distros like build/linux/unbundle that I'd expect to work here, and if it stops working (e.g. incompatible API change) I'm going to figure out the best solution, but that shouldn't affect the project.

Paweł

 

best

-jochen

--

[Emil A Eklund]

> Don't worry though. I think it's fine for the project to move to bundled
> harfbuzz. I'm excited about more improvements to text shaping this would
> enable, and changes to harfbuzz eventually benefit all users of that
> library. As mentioned before, we have existing infrastructure for distros
> like build/linux/unbundle that I'd expect to work here, and if it stops
> working (e.g. incompatible API change) I'm going to figure out the best
> solution, but that shouldn't affect the project.

Yay! Thanks everyone for your input. I'll follow up off-list with
implementation questions.

[Nico Weber]

+1 to never use_system_harfbuzz unconditionally. I just spent some time debugging an issue with tests not running on swarming that was caused by the builder bot running trusty (where things apparently are new enough to use system harfbuzz) and swarming not running trusty (so that the built binary could not run there 'cause system harfbuzz wasn't new enough).

It's ok to keep the flag around if distros want to set it, but the autodetection caused me quite a bit of unnecessary work this morning.

[Christian Biesinger]

What's the easiest way to disable system harfbuzz for my local builds?
I have various failing layout tests that seem to use complex text, and
I'd like to try disabling it to see if that fixes them. Thanks!

-christian

[Nico Weber]

GYP_DEFINES=use_system_harfbuzz=0

[Christian Biesinger]

Thanks!

Sadly that didn't work so well :( Lots of warnings (-> errors) like this:

/src/chromium/src/third_party/binutils/Linux_x64/Release/bin/ld.gold:
warning: hidden symbol 'hb_buffer_create' in
obj/third_party/harfbuzz-ng/libharfbuzz-ng.a(obj/third_party/harfbuzz-ng/src/harfbuzz-ng.hb-buffer.o)
is referenced by DSO
/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/libpangoft2-1.0.so
/src/chromium/src/third_party/binutils/Linux_x64/Release/bin/ld.gold:
error: treating warnings as errors

-christian

[Nico Weber]

Hm, does using bundled harfbuzz require using bundled pango? Do we even have a bundled pangocairo? eae or phajdan, does one of you know how this is supposed to work?

[Emil A Eklund]

On Thu, Mar 5, 2015 at 4:27 PM, Nico Weber <tha...@chromium.org> wrote:
> Hm, does using bundled harfbuzz require using bundled pango? Do we even have
> a bundled pangocairo? eae or phajdan, does one of you know how this is
> supposed to work?

To get it to work today you'll either need to build an official build
or an embedded one. I'll try to find some time to look into what it
would entail to always use the bundled version in a week or two.

[Rottsches, Dominik]

On Thu, 2015-03-05 at 19:10 -0500, Christian Biesinger wrote:
> What's the easiest way to disable system harfbuzz for my local builds?
> I have various failing layout tests that seem to use complex text, and
> I'd like to try disabling it to see if that fixes them. Thanks!

See also
https://code.google.com/p/chromium/issues/detail?id=353127
in which some of the discussion revolving around linking issues and
rationale for chosing the system harfbuzz on Chrome OS is tracked.

@eae: Thanks for looking into always using the bundled version, much
appreciated.

Dominik