PSA: Linux builds to use sysroot image by default

[Sam Clegg]

I'm hoping to land the following change today:
https://codereview.chromium.org/1464983003/

"""
Use sysroot by default for all linux builds

Using a sysroot was already the default for official
builds and when cross compiling (e.g. arm, mips, x86-32).
This changes makes it the default in all cases.

The old behavior of building against local
system libraries can still be achieved by
setting use_sysroot=0
"""

This means that, by default, the chromium build will not depend on or
use any of your locally installed headers or libraries.

[Antoine Labour]

Which sysroot does it use? Where does it come from? Does it match my local distribution?

IOW, is this going to be hell to compile a browser that just works and runs locally?

Antoine

 

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

[Roland McGrath]

It's the same sysroot that the official builders use. It yields
binaries that run everywhere that official installs of Chrome run
(which is more places than a binary built on your workstation without
a sysroot will run).

[Antoine Labour]

Where does it come from? When is it downloaded?

Antoine

[Dirk Pranke]

On Mon, Nov 23, 2015 at 11:18 AM, 'Antoine Labour' via Chromium-dev <chromi...@chromium.org> wrote:

On Mon, Nov 23, 2015 at 9:09 AM, Sam Clegg <s...@chromium.org> wrote:

I'm hoping to land the following change today:
https://codereview.chromium.org/1464983003/

"""
Use sysroot by default for all linux builds

Using a sysroot was already the default for official
builds and when cross compiling (e.g. arm, mips, x86-32).
This changes makes it the default in all cases.

The old behavior of building against local
system libraries can still be achieved by
setting use_sysroot=0
"""

This means that, by default, the chromium build will not depend on or
use any of your locally installed headers or libraries.

Which sysroot does it use? Where does it come from? Does it match my local distribution?

It uses the same sysroot we use for official builds, which will be older than what you have

installed locally but should run fine.

 

IOW, is this going to be hell to compile a browser that just works and runs locally?

No, it should Just Work (unless for some reason you are trying to test things that aren't available 

in the older distros, which is of course problematic for its own reasons).

If it doesn't, we will need to fix things, since obviously we want people to be able to trivially 

build and test things.

[Nico Weber]

https://codereview.chromium.org/1464983003/#msg24

 

When is it downloaded?

runhooks time

[Dirk Pranke]

We build the sysroots and upload them to a google storage bucket. It's downloaded at runhooks

time. There is a ~35-50MB download, which is large-ish, but not the end of the world given that

the git repos and some of the other downloads are larger.

-- Dirk

[Antoine Labour]

Ok, thanks. Seems like yet another point of failure, but we'll see over time.

Antoine 

[Sam Clegg]

The hope is that it will be less error prone over time than
install-build-deps.sh, which it essentially replaces.
install-build-deps.sh has many problems. For example, it requires
re-imaging all of the bots when a dependency is added. It also
requires every developer manually running it locally.

>
> Antoine

[Dirk Pranke]

This should eliminate a class of issues that arise from having a local build that doesn't

match what we actually ship, and anything where we can make the official builds more

like regular builds is a good thing to me (all other things being equal).

If the point of failure you're worried about is google cloud storage, we already 

have that for a bunch of things, including gn, clang, and the nacl toolchains, 

so this change isn't going to make things worse.

-- Dirk

[Nico Weber]

To me, it feels like it should be more robust: We need working sysroots for official builds anyways. For example, when the libffi dep was added recently we forgot to add it to the sysroot at first, and had to file a ticket to make sure install-build-deps.sh gets rerun on every bot, and still some bots got missed -- if we had had the sysroot-by-default thing neither of those problems would have happened: We would've just built a new sysroot instead of adding a package to build-install-deps.sh.

Also, we'll no longer trip about system headers annotating write with IGNORE_RESULT on some machines but not on others.

[Dirk Pranke]

Right :).

-- Dirk

[Michael Spang]

+1 as this versions the system deps, it means we are one step closer to hermetic builds.

  

It can only be less brittle because we are not at the mercy of the idiosyncrasies of the local system.

Michael

[Lei Zhang]

On Mon, Nov 23, 2015 at 11:40 AM, Michael Spang <sp...@chromium.org> wrote:
> +1 as this versions the system deps, it means we are one step closer to
> hermetic builds.
>
> It can only be less brittle because we are not at the mercy of the
> idiosyncrasies of the local system.

That's another good point. Keep in mind developers on non-Debian-based
distros have to come up with their own version of
install-build-deps.sh.

To further answer Antonie's questions, the scripts in
build/linux/sysroot_scripts/ generates the sysroot tarballs. I think
sbc@ will even write some form of documentation to explain the few
steps needed to generate the tarballs. The scripts download .deb files
from Debian/Ubuntu to generate the sysroot. The system's GPG keyring
is used to verify the Packages file from the server, and then the
SHA256s in the Packages file are used to verify the .debs, just like
Debian-based systems do with apt.

[Henrik Grunell]

Is this related issue fixed or will it be?

https://code.google.com/p/chromium/issues/detail?id=242625

/Henrik

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

--

Henrik Grunell | Software Engineer | gru...@google.com | +46 70 676 09 01

[Lei Zhang]

I closed it WontFix. I hope the last couple years of doing local
official Linux builds was not too painful.

[Jeffrey Yasskin]

If you're trying to build with gn for Android, and you get lots of
'file not found' errors, try running:

$ GYP_DEFINES="$GYP_DEFINES OS=android" gclient runhooks

That'll download the i386 sysroot, which the android build seems to
want, rather than the amd64 sysroot that the linux build wants.

I assume this means that for any particular set of environment
variables, either the Android or Linux sysroot will gradually get out
of date?

Jeffrey

On Mon, Nov 23, 2015 at 9:09 AM, Sam Clegg <s...@chromium.org> wrote:

[Dirk Pranke]

On Tue, Nov 24, 2015 at 5:35 PM, Jeffrey Yasskin <jyas...@chromium.org> wrote:

If you're trying to build with gn for Android, and you get lots of
'file not found' errors, try running:

$ GYP_DEFINES="$GYP_DEFINES OS=android" gclient runhooks

That'll download the i386 sysroot, which the android build seems to
want, rather than the amd64 sysroot that the linux build wants.

Technically we want the android build to have both the 32-bit and 64-bit

x86 sysroots as well as the checked-in arm sysroot that we get as part

of the NDK.

 

I assume this means that for any particular set of environment
variables, either the Android or Linux sysroot will gradually get out
of date?

See my other responses to the "Android GN compilation failure, maybe

related to V8" thread for more on the underlying problem. We will

need to fix this when we finally get rid of GYP, if not earlier, since 

"GYP_DEFINES" will not make sense at that point.

Fortunately, the sysroots change once or twice a year so this might not

be an issue past the initial download, if we get lucky :)

-- Dirk

[Satoru Takahashi]

—
Mailbox から送信

[kphanee]

Hi Sam/Dirk,

After making sysroot image default change, my linux build with default configuration was failing. I had to give use_sysroot=0 explicitly to continue the build using system headers.

Though the code checkout was having folders build/linux/debian_wheezy_amd64-sysroot/ and build/linux/debian_wheezy_i386-sysroot/, build was still not picking correctly the standard headers from sysroot location.

Do we need to make a clobber build after the change? Do we need to change the GYP_DEFINES or build environment so that sysroot is picked correctly? Wouldn't the build command be updated to pick sysroot folder instead of system header once gyp_chromium is completed after the change?

My default GYP configuration:

GYP_DEFINES="component=shared_library clang=0"

GYP_GENERATOR_FLAGS=output_dir=out_desktop

Regards,

Ravi Kasibhatla

[Dirk Pranke]

The only thing you should've needed to do to pick up the change was to run

`gclient sync` or `gclient runhooks`. No need to change GYP_DEFINES.

If the sysroots did get downloaded but the build didn't see them, I don't have an

immediate explanation. Possibly re-running build/gyp_chromium would work,

but it shouldn't have been needed.

If the problem persists, maybe file a bug and we can investigate further there?

-- Dirk

[Dirk Pranke]

Actually it looks like the sysroots may not work w/ newer gcc's (which you're getting w/ use_clang=0).

See https://code.google.com/p/chromium/issues/detail?id=504446#c17

So you're probably stuck w/ not using the sysroot for now.

-- Dirk

[Ravi Kasibhatla]

Ah, IIRC my current gcc version is 4.8.3. Can we default use_sysroot=0 when using clang=0 to avoid these errors?

[Dirk Pranke]

I think that should be fine. Feel free to post a patch for that and we can review.

-- Dirk