How to build chromium-lite?

[Zoltan Boszormenyi]

I am moving the problem from the chromium list.

UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

Steps to reproduce the problem:
1. Download http://gsdview.appspot.com/chromium-browser-official/chromium-51.0.2704.63-lite.tar.xz

2. Try to build it with these flags:

$ cat include.gypi 
{
  'variables': {
    'target_arch': 'x64',
  }
}

$ cat oe-defaults.gypi 
{
  'variables': {
    'use_system_harfbuzz': 1,
    'use_system_fontconfig':1,
    'use_system_libjpeg':1,
    'use_libjpeg_turbo':1,
    'use_system_libxml':1,
    'use_system_zlib':1,
    'use_system_expat':1,
    'use_system_libexif':1,
    'disable_nacl': 1,
    'proprietary_codecs': 1,
    'v8_use_external_startup_data':1,
    'v8_use_snapshot': 'true',
    'use_system_ffmpeg': 0,
    'ffmpeg_branding': 'Chrome',
    'media_use_ffmpeg': 1,
    'media_use_libvpx': 1,
    'media_use_libwebm': 1,
    'linux_use_tcmalloc': 1,
    'linux_link_kerberos': 0,
    'use_kerberos': 0,
    'use_cups': 1,
    'use_gnome_keyring': 0,
    'linux_link_gnome_keyring': 0,
    'use_gconf': 0,
    'use_gio': 0,
  }, 
}

GYP_DEFINES='-Dangle_use_commit_id=0 -Dclang=0 -Dhost_clang=0 -Ddisable_fatal_linker_warnings=1 -Dlinux_use_gold_binary=0 -Dlinux_use_gold_flags=0 -Dwant_separate_host_toolset=0 -Duse_system_icu=1 -Dicu_use_data_file_flag=0 -Dv8_enable_vtunejit=0 -Dtest_isolation_mode="noop" '

3. Start build.

What is the expected behavior?
I don't know.

What went wrong?
Updating projects from gyp files...
Traceback (most recent call last):
  File "build/gyp_chromium", line 12, in <module>
    execfile(__file__ + '.py')
  File "build/gyp_chromium.py", line 341, in <module>
    sys.exit(main())
  File "build/gyp_chromium.py", line 328, in main
    gyp_rc = gyp.main(args)
  File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6/chromium-51.0.2704.63/tools/gyp/pylib/gyp/__init__.py", line 538, in main
    return gyp_main(args)
  File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6/chromium-51.0.2704.63/tools/gyp/pylib/gyp/__init__.py", line 514, in gyp_main
    options.duplicate_basename_check)
  File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6/chromium-51.0.2704.63/tools/gyp/pylib/gyp/__init__.py", line 130, in Load
    params['parallel'], params['root_targets'])
  File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6/chromium-51.0.2704.63/tools/gyp/pylib/gyp/input.py", line 2800, in Load
    RemoveLinkDependenciesFromNoneTargets(targets)
  File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6/chromium-51.0.2704.63/tools/gyp/pylib/gyp/input.py", line 1510, in RemoveLinkDependenciesFromNoneTargets
    if targets[t].get('variables', {}).get('link_dependency', 0):
KeyError: '/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6/chromium-51.0.2704.63/v8/src/d8.gyp:d8#host'

Did this work before? N/A 

Chrome version: 51.0.2704.63  Channel: n/a
OS Version: 
Flash Version: 

I am running this through a Bitbake recipe for Yocto to build chromium-lite but it shouldn't be a problem. The big brother, Chromium from the same version builds fine, except that that last few flags are not needed for it, because it contains the ICU and the vtunejit 3rd party codes built-in. These flags are omitted for Chromium:

-Dwant_separate_host_toolset=0 -Duse_system_icu=1 -Dicu_use_data_file_flag=0 -Dv8_enable_vtunejit=0 -Dtest_isolation_mode="noop" '

What else would be needed or what should be changed to make chromium-lite tree build?

[Paweł Hajdan, Jr.]

What are you trying to do?

Note that chromium-lite tarballs are just like chromium tarballs with some files removed so they are smaller. However, they're "maintained" on best-effort basis, and might have some rot.

Also note that with many custom build flags, you shouldn't expect any support even here. I usually try to give people at least some hints, but again, this is mostly best-effort.

Paweł

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Böszörményi Zoltán]

2016-07-13 09:39 keltezéssel, Paweł Hajdan, Jr. írta:
> What are you trying to do?

The end goal is to to run it on a device with only 2GB RAM that cannot be upgraded. POS
hardware...
The real chromium often dies with "It's dead, Jim" or something like that. Any space
saving would help.

I also noticed that Chromium doesn't like if I disable memory overcommit in Linux,
it crashes very soon afterward. That would be the biggest improvement if chromium
did not over-allocate.

>
> Note that chromium-lite tarballs are just like chromium tarballs with some files removed
> so they are smaller.

I noticed that. The build framework seems to be not adjusted for the removals.

> However, they're "maintained" on best-effort basis, and might have some rot.
>

> Also note that with many custom build flags, you shouldn't /expect/ any support even

> here. I usually try to give people at least some hints, but again, this is mostly
> best-effort.

I have tried to report it as a "bug" that was closed yesterday as "wontfix" and
they suggested to use the chromium-dev list for build questions.

There are no custom build flags in the build really, only that are in the gyp files in
Chromium.

Without the extra flags for chromium-lite that were quoted below,
the build setup phase died even earlier, so the bitrot may actually be true.
Why is chromium-lite published at all if its build isn't tested?

Thanks anyway.

>
> Paweł
>
> On Wed, Jul 13, 2016 at 7:18 AM, Zoltan Boszormenyi <zbo...@gmail.com
> <mailto:zbo...@gmail.com>> wrote:
>
> I am moving the problem from the chromium list.
>
> UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36
>
> Steps to reproduce the problem:

> 1. Downloadhttp://gsdview.appspot.com/chromium-browser-official/chromium-51.0.2704.63-lite.tar.xz

> File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6 <https://crrev.com/6>/chromium-51.0.2704.63/tools/gyp/pylib/gyp/__init__.py", line 538, in main
> return gyp_main(args)
> File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6 <https://crrev.com/6>/chromium-51.0.2704.63/tools/gyp/pylib/gyp/__init__.py", line 514, in gyp_main
> options.duplicate_basename_check)
> File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6 <https://crrev.com/6>/chromium-51.0.2704.63/tools/gyp/pylib/gyp/__init__.py", line 130, in Load

> params['parallel'], params['root_targets'])

> File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6 <https://crrev.com/6>/chromium-51.0.2704.63/tools/gyp/pylib/gyp/input.py", line 2800, in Load
> RemoveLinkDependenciesFromNoneTargets(targets)
> File "/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6 <https://crrev.com/6>/chromium-51.0.2704.63/tools/gyp/pylib/gyp/input.py", line 1510, in RemoveLinkDependenciesFromNoneTargets

> if targets[t].get('variables', {}).get('link_dependency', 0):

> KeyError: '/data/OE/setup-scripts/build/tmp-angstrom_v2014_06-eglibc/work/core2-64-sicom-linux/chromium-lite/51.0.2704.63-r6 <https://crrev.com/6>/chromium-51.0.2704.63/v8/src/d8.gyp:d8#host'

>
> Did this work before? N/A
>
> Chrome version: 51.0.2704.63 Channel: n/a
> OS Version:
> Flash Version:
>
> I am running this through a Bitbake recipe for Yocto to build chromium-lite but it shouldn't be a problem. The big brother, Chromium from the same version builds fine, except that that last few flags are not needed for it, because it contains the ICU and the vtunejit 3rd party codes built-in. These flags are omitted for Chromium:
>
> -Dwant_separate_host_toolset=0 -Duse_system_icu=1 -Dicu_use_data_file_flag=0 -Dv8_enable_vtunejit=0 -Dtest_isolation_mode="noop" '
>
> What else would be needed or what should be changed to make chromium-lite tree build?
>
> --
> --
> Chromium Developers mailing list: chromi...@chromium.org

> <mailto:chromi...@chromium.org>

[Paweł Hajdan, Jr.]

On Wed, Jul 13, 2016 at 7:11 PM, Böszörményi Zoltán <zbo...@gmail.com> wrote:

2016-07-13 09:39 keltezéssel, Paweł Hajdan, Jr. írta:

What are you trying to do?

The end goal is to to run it on a device with only 2GB RAM that cannot be upgraded. POS hardware...
The real chromium often dies with "It's dead, Jim" or something like that. Any space saving would help.

chromium-lite tarballs won't help with that. The only reason they exist is that tarball itself is smaller.

If there's something you don't compile from the smaller tarball, it'll also be true for the larger tarball. I don't see a way to have any difference in compiled code between the two.

 

I also noticed that Chromium doesn't like if I disable memory overcommit in Linux,
it crashes very soon afterward. That would be the biggest improvement if chromium
did not over-allocate.

I'm not sure if we considered making the code work when memory overcommit is disabled. If you have more specific suggestions or patches, they can be discussed here (possibly on another thread).

 

Note that chromium-lite tarballs are just like chromium tarballs with some files removed so they are smaller.

I noticed that. The build framework seems to be not adjusted for the removals.

Yes. One is supposed to use build/linux/unbundle, but again, it just breaks from time to time and is only updated when someone has time to do so.

 

However, they're "maintained" on best-effort basis, and might have some rot.

Also note that with many custom build flags, you shouldn't /expect/ any support even here. I usually try to give people at least some hints, but again, this is mostly best-effort.

I have tried to report it as a "bug" that was closed yesterday as "wontfix" and
they suggested to use the chromium-dev list for build questions.

Feel free to post the bug link here so we can have something more specific.

 

There are no custom build flags in the build really, only that are in the gyp files in Chromium.

This may be non-obvious: the reality is that even if they exist, it doesn't mean they're supported. It doesn't mean they work. It doesn't mean every combination is supported and works. Some of these might be legitimate bugs, but I think it'd be decided on case-by-case basis, and you'd need to untangle just 2-3 specific flags rather than ~20.

 

Without the extra flags for chromium-lite that were quoted below,
the build setup phase died even earlier, so the bitrot may actually be true.
Why is chromium-lite published at all if its build isn't tested?

We don't really advertise it, so people who tend to know about it usually accept the caveats.

Paweł

[Primiano Tucci]

One thing you could try is the --enable-low-end-device-mode. I have no idea if it works at all on desktop (it is designed for Android), but might be worth a try.

But I think no overcommit is really what kills you here. V8 can easily mmap pools of hundreds of MB per renderer process. similarly for Blink GC I suppose.

On Thu, Jul 14, 2016 at 9:08 AM Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:

I also noticed that Chromium doesn't like if I disable memory overcommit in Linux,
it crashes very soon afterward. That would be the biggest improvement if chromium
did not over-allocate.

I'm not sure if we considered making the code work when memory overcommit is disabled. If you have more specific suggestions or patches, they can be discussed here (possibly on another thread).

+1 to Pawel. I personally think it's an unnecessary and counter productive goal.

Chrome these days has at least two garbage collection systems (V8 and Oilpan).

Over-allocating virtual memory not only makes that code easier to maintain but also more performant. The alternative would be doing mmap syscalls more frequently (mmap is quite expensive, takes the map_sem, blocks all concurrent page fault) and doing more bookeping locally.

Most non-trivial codebases these days will just explode if you disable overcommit. It is just too convenient to reason in terms of having a huge pool of virtually contiguous space.

Why are you disabling overcommit at all? It seems that your use case is a device with quite limited memory constraints, which is one of the reasons why overcommit was introduced in the first place.

[Böszörményi Zoltán]

https://bugs.chromium.org/p/chromium/issues/detail?id=615631

No more details there, the whole text was copy&pasted into this thread's starting mail.

> There are no custom build flags in the build really, only that are in the gyp files
> in Chromium.
>
>
> This may be non-obvious: the reality is that even if they exist, it doesn't mean they're
> supported. It doesn't mean they work. It doesn't mean every combination is supported and
> works. Some of these might be legitimate bugs, but I think it'd be decided on
> case-by-case basis, and you'd need to untangle just 2-3 specific flags rather than ~20.
>
> Without the extra flags for chromium-lite that were quoted below,
> the build setup phase died even earlier, so the bitrot may actually be true.
> Why is chromium-lite published at all if its build isn't tested?
>
>
> We don't really advertise it, so people who tend to know about it usually accept the
> caveats.
>
> Paweł
>
> Thanks anyway.
>
>
> Paweł
>
> On Wed, Jul 13, 2016 at 7:18 AM, Zoltan Boszormenyi <zbo...@gmail.com

> <mailto:zbo...@gmail.com> <mailto:zbo...@gmail.com

> <mailto:chromi...@chromium.org <mailto:chromi...@chromium.org>>