GN changes landed

321 views
Skip to first unread message

Brett Wilson

unread,
Jan 17, 2014, 5:33:34 PM1/17/14
to Chromium-dev
I just converted the canonical version of the third_party/re2 target
to be GN and it seems to be sticking this time.

This means that when you run GYP, it will first run GN which will
convert src/third_party/re2/BUILD.gn into
out/gn_gyp/third_party/re2/re2.gyp. If you add new dependencies on
this target, you'll need to code the location of this GYP file:
http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser.gypi?r1=245563&r2=245562&pathrev=245563

As we add more stuff, more GYP files will appear in the out/gn_gyp
mirror tree, so please be aware that you may need to look there for
some targets. I don't want to write generated files to the main tree.
We can consider adding automatic fallback to this mirror tree to GYP
if it proves too annoying.

Recall there is documentation here:
https://code.google.com/p/chromium/wiki/gn including a quick start for
making a hello world target. I'll be in-and-out of the office for the
next few weeks but will be available to answer email if you have any
questions and will be working on fixing some of the issues.

--

This means you can now convert targets to GN without having to
maintain the GYP build in parallel. Currently, the best thing is to
find a static_library leaf-node (like in third_party).

Stuff that doesn't work now:

- Things that say 'toolsets': ['host'] in the target. There may or may
not be a "target" in that list as well. I haven't hooked up the "host"
cross-compile on Linux yet.

- Currently, only static libraries can be converted because there is
some linker stuff for Android that I still haven't hooked up for
shared libraries and executables. I hope to get this done soon.

- Running scripts aren't supported in the GYP output mode yet. So
don't try to convert any GYP actions.

--

There are some known issues. Since this is using the binary much more
than previously, there are some bugs. I know about a lock assertion on
Windows and a double-free on Linux that happen from time to time. They
generally will print a stack trace as part of running GYP, in a way
that is confusing where the error is actually coming from (due to
output buffering issues). I'll try to get these figured out soon.

In the meantime, if you see a stack trace and failure when running
hooks on your local machine or a buildbot, just try again.

There is some warnings spew from the re2. Feel free to look at which
warnings need to be disabled!

If you see an error about use_aura, being doubly-set (independent of
today's patch, this went in last week), remove use_aura from your
GYP_DEFINES. It's now redundant on a Windows build which is what this
error is telling you. I have a patch that removes the warning in this
case since it doesn't seem to be helping, but I haven't gotten the new
binary pushed yet.

Brett

Dirk Pranke

unread,
Jan 17, 2014, 5:35:58 PM1/17/14
to Brett Wilson, Chromium-dev
Woo! 

-- Dirk



--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev

Nico Weber

unread,
Jan 17, 2014, 5:36:18 PM1/17/14
to Brett Wilson, Chromium-dev


On Fri, Jan 17, 2014 at 2:33 PM, Brett Wilson <bre...@chromium.org> wrote:

Nico Weber

unread,
Jan 17, 2014, 5:37:58 PM1/17/14
to Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 2:36 PM, Nico Weber <tha...@chromium.org> wrote:

After looking for more than a few seconds, that seems to be a complete list of broken stuff that's reachable from the main waterfall. So "lots" == "more than 1" :-)

Antoine Labour

unread,
Jan 17, 2014, 5:38:31 PM1/17/14
to Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 2:33 PM, Brett Wilson <bre...@chromium.org> wrote:
I just converted the canonical version of the third_party/re2 target
to be GN and it seems to be sticking this time.

This means that when you run GYP, it will first run GN which will
convert src/third_party/re2/BUILD.gn into
out/gn_gyp/third_party/re2/re2.gyp.

To be clear, since this ignores output_dir, this generation doesn't depend on the GYP_DEFINES or other environment state, is that correct?

Antoine
 
--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

Ami Fischman

unread,
Jan 17, 2014, 5:44:14 PM1/17/14
to Antoine Labour, Brett Wilson, Chromium-dev
No, I don't think that's right, and I think that's a bug.
(as an example, --sysroot is embedded in ldflags in the emitted re2.gyp for my chrome/android set of GYP_DEFINES, but -m64 is emitted for my cros set of GYP_DEFINES).

This breaks the use-case of using the same chromium checkout for development on multiple platforms, or of development with different-enough sets of GYP_DEFINES.

Cheers,
-a

Nico Weber

unread,
Jan 17, 2014, 5:48:15 PM1/17/14
to Ami Fischman, Antoine Labour, Brett Wilson, Chromium-dev
Ami is correct. gn currently generated gyp files, and neither gn and gyp ignore GYP_DEFINES. Since gn ignores -Goutput_dir, it'll clobber its old output every time, likely leaving you in a broken state.

James Robinson

unread,
Jan 17, 2014, 5:51:51 PM1/17/14
to Ami Fischman, Antoine Labour, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 2:44 PM, Ami Fischman <fisc...@chromium.org> wrote:
No, I don't think that's right, and I think that's a bug.
(as an example, --sysroot is embedded in ldflags in the emitted re2.gyp for my chrome/android set of GYP_DEFINES, but -m64 is emitted for my cros set of GYP_DEFINES).

This breaks the use-case of using the same chromium checkout for development on multiple platforms, or of development with different-enough sets of GYP_DEFINES.

The way to do this in a pure gn environment would be to configure different outs for different configs (debug, release, asan, android) and then have ninja pick up the defines for each one based on the setup.  You're right that there is no way to do this in the gn-gyp hybrid.  I think I'll just figure out another workflow to use for mixing my android/etc builds with my chromeos one - maybe by switching a symlink for 'out' around or something.  It's a relatively niche use case and I do not think there's a great alternative in this intermediate state.

- James


Brett Wilson

unread,
Jan 17, 2014, 5:55:42 PM1/17/14
to Ami Fischman, Antoine Labour, Chromium-dev
On Fri, Jan 17, 2014 at 2:44 PM, Ami Fischman <fisc...@chromium.org> wrote:
> No, I don't think that's right, and I think that's a bug.
> (as an example, --sysroot is embedded in ldflags in the emitted re2.gyp for
> my chrome/android set of GYP_DEFINES, but -m64 is emitted for my cros set of
> GYP_DEFINES).
>
> This breaks the use-case of using the same chromium checkout for development
> on multiple platforms, or of development with different-enough sets of
> GYP_DEFINES.

I'm not very familiar with that workflow. I think the only way to make
that work would be to teach GYP about the magic secondary tree of
generated GYP files. Unless somebody knows how to reference a file
relative to the -Goutput thing from GYP files.

If you do this kind of thing, in the meantime you can just run
build/gyp_chromium again.

Brett

Nico Weber

unread,
Jan 17, 2014, 5:56:55 PM1/17/14
to James Robinson, Ami Fischman, Antoine Labour, Brett Wilson, Chromium-dev
Just let gyp_chromium pass -Goutput_dir on to gn, no?
 

- James


James Robinson

unread,
Jan 17, 2014, 5:58:21 PM1/17/14
to Nico Weber, Ami Fischman, Antoine Labour, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 2:56 PM, Nico Weber <tha...@chromium.org> wrote:
On Fri, Jan 17, 2014 at 2:51 PM, James Robinson <jam...@google.com> wrote:
On Fri, Jan 17, 2014 at 2:44 PM, Ami Fischman <fisc...@chromium.org> wrote:
No, I don't think that's right, and I think that's a bug.
(as an example, --sysroot is embedded in ldflags in the emitted re2.gyp for my chrome/android set of GYP_DEFINES, but -m64 is emitted for my cros set of GYP_DEFINES).

This breaks the use-case of using the same chromium checkout for development on multiple platforms, or of development with different-enough sets of GYP_DEFINES.

The way to do this in a pure gn environment would be to configure different outs for different configs (debug, release, asan, android) and then have ninja pick up the defines for each one based on the setup.  You're right that there is no way to do this in the gn-gyp hybrid.  I think I'll just figure out another workflow to use for mixing my android/etc builds with my chromeos one - maybe by switching a symlink for 'out' around or something.  It's a relatively niche use case and I do not think there's a great alternative in this intermediate state.

Just let gyp_chromium pass -Goutput_dir on to gn, no?

How would you specify the dependency on the gn-generated gyp file from the checked-in gyp file?  Have a gyp variable <(output_dir) or something?

- James

Brett Wilson

unread,
Jan 17, 2014, 6:02:01 PM1/17/14
to Nico Weber, James Robinson, Ami Fischman, Antoine Labour, Chromium-dev
On Fri, Jan 17, 2014 at 2:56 PM, Nico Weber <tha...@chromium.org> wrote:
> On Fri, Jan 17, 2014 at 2:51 PM, James Robinson <jam...@google.com> wrote:
>>
>> On Fri, Jan 17, 2014 at 2:44 PM, Ami Fischman <fisc...@chromium.org>
>> wrote:
>>>
>>> No, I don't think that's right, and I think that's a bug.
>>> (as an example, --sysroot is embedded in ldflags in the emitted re2.gyp
>>> for my chrome/android set of GYP_DEFINES, but -m64 is emitted for my cros
>>> set of GYP_DEFINES).
>>>
>>> This breaks the use-case of using the same chromium checkout for
>>> development on multiple platforms, or of development with different-enough
>>> sets of GYP_DEFINES.
>>
>>
>> The way to do this in a pure gn environment would be to configure
>> different outs for different configs (debug, release, asan, android) and
>> then have ninja pick up the defines for each one based on the setup. You're
>> right that there is no way to do this in the gn-gyp hybrid. I think I'll
>> just figure out another workflow to use for mixing my android/etc builds
>> with my chromeos one - maybe by switching a symlink for 'out' around or
>> something. It's a relatively niche use case and I do not think there's a
>> great alternative in this intermediate state.
>
>
> Just let gyp_chromium pass -Goutput_dir on to gn, no?

No, the harder part is that all the .gyp files need to refer to the
generated files, which potentially means we need the variable that
James was talking about in GYP (unless there is one already).

Brett

Nico Weber

unread,
Jan 17, 2014, 6:06:09 PM1/17/14
to Brett Wilson, James Robinson, Ami Fischman, Antoine Labour, Chromium-dev
build/gyp_chromium can just -Dgn_compat_hack=out_foo, no?
 

Brett

Ami Fischman

unread,
Jan 17, 2014, 6:07:55 PM1/17/14
to James Robinson, Nico Weber, Antoine Labour, Brett Wilson, Chromium-dev
How would you specify the dependency on the gn-generated gyp file from the checked-in gyp file?  Have a gyp variable <(output_dir) or something?

gyp 'include' resolution doesn't use variable expansion so this would only work for dependencies, not includes, unless this was a magical new kind of variable that was expanded for includes (previously discussed & rejected on gyp-developer for other use-cases).

Maybe teach gyp about a "shadow" tree to use for looking up gyps on failure to find them in the source tree?

(I don't see a way forward here that doesn't involve extending gyp)

-a

Antoine Labour

unread,
Jan 17, 2014, 6:08:38 PM1/17/14
to Nico Weber, Brett Wilson, James Robinson, Ami Fischman, Chromium-dev
It can, but it needs to extract the output_dir out of the command-line argument and/or GYP_GENERATOR_FLAGS
Or we could fix it in gyp, I suppose - we already reference variables that include output_dir (e.g. INTERMEDIATE_DIR etc.)

Antoine

 

Brett


James Robinson

unread,
Jan 17, 2014, 6:11:18 PM1/17/14
to Ami Fischman, Nico Weber, Antoine Labour, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 3:07 PM, Ami Fischman <fisc...@chromium.org> wrote:
How would you specify the dependency on the gn-generated gyp file from the checked-in gyp file?  Have a gyp variable <(output_dir) or something?

gyp 'include' resolution doesn't use variable expansion so this would only work for dependencies, not includes, unless this was a magical new kind of variable that was expanded for includes (previously discussed & rejected on gyp-developer for other use-cases).

Sure, but you would never want to 'include' a gn-generated file since gn never generates gypi files, only fully-resolved .gyp files.  So I don't think this would be a problem.

- James

Antoine Labour

unread,
Jan 17, 2014, 6:11:45 PM1/17/14
to James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 2:51 PM, James Robinson <jam...@google.com> wrote:
If "niche" = develops in more than one config, that most likely includes everyone who develops for hard-to-debug-on devices (e.g. Android, Chrome OS).

Antoine


- James



Ami Fischman

unread,
Jan 17, 2014, 6:17:27 PM1/17/14
to James Robinson, Nico Weber, Antoine Labour, Brett Wilson, Chromium-dev
I think you know a thing that I do not know about .gypi files: what does "fully-resolved" mean for .gyp files?
Perhaps put another way: is apps/apps.gypi "fully-resolved"?

Maybe the answer is that include trees need to be migrated to the gn-gyp build atomically, and use gn's import facility?

James Robinson

unread,
Jan 17, 2014, 6:18:35 PM1/17/14
to Antoine Labour, Ami Fischman, Brett Wilson, Chromium-dev
I know you build multiple configs from the same checkout, and I do, and Nico does, but truth is the vast majority of developers build for one config.  Of the developers who build for multiple configs, the vast majority of these have different checkouts for each config.  That's what the documentation says to do, after all.  It's only a small minority that actually builds multiple configs from the same checkout.

Anyway, it sounds like the gyp variable thingy will work so let's do that.

- James


Antoine


- James




Nico Weber

unread,
Jan 17, 2014, 6:20:00 PM1/17/14
to James Robinson, Antoine Labour, Ami Fischman, Brett Wilson, Chromium-dev
I know you build multiple configs from the same checkout, and I do, and Nico does, but truth is the vast majority of developers build for one config.  Of the developers who build for multiple configs, the vast majority of these have different checkouts for each config.  That's what the documentation says to do, after all.  It's only a small minority that actually builds multiple configs from the same checkout. [1]

Filed http://crbug.com/335760 for getting this working again.

[1] citation needed; also I hope to increase this by making it easier to build multiple configs so this is kinda moving in the wrong direction right now
 

Anyway, it sounds like the gyp variable thingy will work so let's do that.

- James


Antoine


- James




Antoine Labour

unread,
Jan 17, 2014, 6:20:26 PM1/17/14
to James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
FWIW, the chromeos_chrome ebuild uses output_dir.

Antoine

Haixia Shi

unread,
Jan 17, 2014, 6:24:23 PM1/17/14
to Antoine Labour, James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
This link error starts to show up when building for chrome OS ARMv7a targets (already removed my out dir and did a clean build):

/Work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5245.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22/ld.gold.real.elf: error: obj/out/gn_gyp/third_party/re2/libre2.a(obj/out/gn_gyp/third_party/re2/../../../../third_party/re2/re2/re2.re2.o) uses VFP register arguments, output does not


Brett Wilson

unread,
Jan 17, 2014, 6:29:15 PM1/17/14
to Ami Fischman, James Robinson, Nico Weber, Antoine Labour, Chromium-dev
On Fri, Jan 17, 2014 at 3:17 PM, Ami Fischman <fisc...@chromium.org> wrote:
> I think you know a thing that I do not know about .gypi files: what does
> "fully-resolved" mean for .gyp files?
> Perhaps put another way: is apps/apps.gypi "fully-resolved"?

I added this concept to GYP for the GN-generated files. It means that
no includes are processed, including common.gypi. This is so GN can
specify the exact set of sources and flags for the given platform that
result from running the GN script and won't fight with all the
conditions and overrides in common.gypi. If you look at the generated
file, you can see it's exceedingly more verbose than the regular ones
(but probably faster to execute in GYP due to lack of dictionary
merges and conditions).

Brett

Ami Fischman

unread,
Jan 17, 2014, 6:31:51 PM1/17/14
to Brett Wilson, James Robinson, Nico Weber, Antoine Labour, Chromium-dev
Thanks Brett.  So it's not supposed to be possible to migrate gyp files which are included into other ones independently, rather each connected component in the include graph must be migrated as a whole to the gn-gyp build?
(I don't have a problem with that, just trying to figure out the parameters of the new world)

Brett Wilson

unread,
Jan 17, 2014, 6:38:14 PM1/17/14
to Ami Fischman, James Robinson, Nico Weber, Antoine Labour, Chromium-dev
On Fri, Jan 17, 2014 at 3:31 PM, Ami Fischman <fisc...@chromium.org> wrote:
> Thanks Brett. So it's not supposed to be possible to migrate gyp files
> which are included into other ones independently, rather each connected
> component in the include graph must be migrated as a whole to the gn-gyp
> build?
> (I don't have a problem with that, just trying to figure out the parameters
> of the new world)

Kind of, but in some sense the question doesn't make sense.

You don't migrate files, you migrate "targets". One generator backend
for GN is one that writes GYP files. So what comes out are .gyp files
with lists of targets and flags in them. How you ended up with a
particular target in GYP originally is irrelevant to the GN build,
you'll end up rewriting all of it GN.

Brett

Antoine Labour

unread,
Jan 17, 2014, 6:42:38 PM1/17/14
to Haixia Shi, James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 3:24 PM, Haixia Shi <hs...@chromium.org> wrote:
This link error starts to show up when building for chrome OS ARMv7a targets (already removed my out dir and did a clean build):

/Work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5245.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22/ld.gold.real.elf: error: obj/out/gn_gyp/third_party/re2/libre2.a(obj/out/gn_gyp/third_party/re2/../../../../third_party/re2/re2/re2.re2.o) uses VFP register arguments, output does not

That brings another point, which is that since GN and GYP use different logic to look at flags / find the compiler / build the command line (and GN is strongly opinionated about those), is there a way to audit, as we do the conversion, that we're not dropping flags / switching compilers under the hood? That's a good way to introduce hard-to-find regressions.

Antoine

Haixia Shi

unread,
Jan 17, 2014, 6:43:27 PM1/17/14
to Antoine Labour, James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
Follow-up on the link failure for ARM: it appears the generated out/gn_gyp/third_party/re2/re2.gyp specifies '-mfloat-abi=softfp' while the rest of chrome uses "-mfloat-abi=hard".

Brett Wilson

unread,
Jan 17, 2014, 6:45:32 PM1/17/14
to Haixia Shi, Antoine Labour, James Robinson, Ami Fischman, Chromium-dev
On Fri, Jan 17, 2014 at 3:24 PM, Haixia Shi <hs...@chromium.org> wrote:
> This link error starts to show up when building for chrome OS ARMv7a targets
> (already removed my out dir and did a clean build):
>
> /Work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5245.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22/ld.gold.real.elf:
> error:
> obj/out/gn_gyp/third_party/re2/libre2.a(obj/out/gn_gyp/third_party/re2/../../../../third_party/re2/re2/re2.re2.o)
> uses VFP register arguments, output does not

Is there a bot for this that I can look at? If not, what are your
GYP_DEFINES (and any other include.gypi stuff you might have)?

Brett

Haixia Shi

unread,
Jan 17, 2014, 6:46:57 PM1/17/14
to Brett Wilson, Antoine Labour, James Robinson, Ami Fischman, Chromium-dev
Not sure about bots but here's my GYP_DEFINES

remoting=1 branding=Chrome v8_use_arm_eabi_hardfloat=true use_vtable_verify=0 use_brlapi=1 arm_neon=1 chromeos=1 target_arch=arm use_xi2_mt=2 v8_can_use_vfp3_instructions=false swig_defines=-DOS_CHROMEOS release_extra_cflags=-g arm_float_abi=hard linux_link_libbrlapi=1 armv7=1 use_cras=1 proprietary_codecs=1 sysroot=/Work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5245.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz system_libdir=lib buildtype=Official linux_sandbox_path=/opt/google/chrome/chrome-sandbox arm_fpu=neon v8_can_use_unaligned_accesses=true python_ver=2.7 strip_tests=1 linux_use_tcmalloc=1 pkg-config=pkg-config-daisy

Brett Wilson

unread,
Jan 17, 2014, 6:48:13 PM1/17/14
to Nico Weber, Chromium-dev
On Fri, Jan 17, 2014 at 2:36 PM, Nico Weber <tha...@chromium.org> wrote:
> There are still lots of red bots linked from the main waterfall that I can
> see after looking for a few seconds, for example:
> http://build.chromium.org/p/chromium.memory.fyi/builders/Chromium%20Windows%20Builder%20(DrMemory)
> http://build.chromium.org/p/chromium.memory.fyi/builders/Windows%20Tests%20%28tsan%29/builds/17336

I believe this patch: https://codereview.chromium.org/139783012/ will
fix both of those issues. I'm slightly worried about causing other
problems and since these are just FYI bots, I'm waiting for a try run
before checking in.

Brett

Nico Weber

unread,
Jan 17, 2014, 6:49:45 PM1/17/14
to Brett Wilson, Chromium-dev
Note that this is the official memory waterfall, with memory sheriff rotation and all. "memory.fyi" is a terrible name, but nobody could think of a better name. (The regular "memory" waterfall is part of the main waterfall and is covered by the regular tree sheriffs.) These aren't "just FYI" bots.

Brett Wilson

unread,
Jan 17, 2014, 6:52:20 PM1/17/14
to Nico Weber, Chromium-dev
I can land it whenever if somebody need it, but there is some chance
of breaking stuff on the main waterfall with my fix.

Brett

Antoine Labour

unread,
Jan 17, 2014, 7:05:09 PM1/17/14
to Haixia Shi, James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 3:42 PM, Antoine Labour <pi...@google.com> wrote:



On Fri, Jan 17, 2014 at 3:24 PM, Haixia Shi <hs...@chromium.org> wrote:
This link error starts to show up when building for chrome OS ARMv7a targets (already removed my out dir and did a clean build):

/Work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5245.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22/ld.gold.real.elf: error: obj/out/gn_gyp/third_party/re2/libre2.a(obj/out/gn_gyp/third_party/re2/../../../../third_party/re2/re2/re2.re2.o) uses VFP register arguments, output does not

That brings another point, which is that since GN and GYP use different logic to look at flags / find the compiler / build the command line (and GN is strongly opinionated about those), is there a way to audit, as we do the conversion, that we're not dropping flags / switching compilers under the hood? That's a good way to introduce hard-to-find regressions.

Antoine

Actually... should we remove any config option from common.gypi that is not supported by gn? Once we migrate anything to gn, the option stops working, essentially, so we should remove it.

Stéphane Marchesin

unread,
Jan 17, 2014, 7:12:08 PM1/17/14
to bre...@chromium.org, Chromium-dev
On Fri, Jan 17, 2014 at 2:33 PM, Brett Wilson <bre...@chromium.org> wrote:
> I just converted the canonical version of the third_party/re2 target
> to be GN and it seems to be sticking this time.
>
> This means that when you run GYP, it will first run GN which will
> convert src/third_party/re2/BUILD.gn into
> out/gn_gyp/third_party/re2/re2.gyp. If you add new dependencies on
> this target, you'll need to code the location of this GYP file:
> http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser.gypi?r1=245563&r2=245562&pathrev=245563

Hi Brett,

it seems like this breaks the build for Chrome OS ARM with simple
chrome. It specifically complains about VFP in third_party/re2. Any
idea?

Stéphane

Antoine Labour

unread,
Jan 17, 2014, 7:27:45 PM1/17/14
to Haixia Shi, James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
On Fri, Jan 17, 2014 at 3:42 PM, Antoine Labour <pi...@google.com> wrote:



On Fri, Jan 17, 2014 at 3:24 PM, Haixia Shi <hs...@chromium.org> wrote:
This link error starts to show up when building for chrome OS ARMv7a targets (already removed my out dir and did a clean build):

/Work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5245.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22/ld.gold.real.elf: error: obj/out/gn_gyp/third_party/re2/libre2.a(obj/out/gn_gyp/third_party/re2/../../../../third_party/re2/re2/re2.re2.o) uses VFP register arguments, output does not

That brings another point, which is that since GN and GYP use different logic to look at flags / find the compiler / build the command line (and GN is strongly opinionated about those), is there a way to audit, as we do the conversion, that we're not dropping flags / switching compilers under the hood? That's a good way to introduce hard-to-find regressions.

Antoine

Practically, here's how a file from base is compiled with a similar  setup:
/usr/local/google/home/piman/goma-cros/armv7a-cros-linux-gnueabi-g++ -B/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22-gold -MMD -MF obj/base/x11/base.edid_parser_x11.o.d -DV8_DEPRECATION_WARNINGS -D_FILE_OFFSET_BITS=64 -DGOOGLE_CHROME_BUILD -DENABLE_RLZ -DTOOLKIT_VIEWS=1 -DUI_COMPOSITOR_IMAGE_TRANSPORT -DUSE_AURA=1 -DUSE_ASH=1 -DUSE_CAIRO=1 -DUSE_CRAS=1 -DUSE_GLIB=1 -DUSE_DEFAULT_RENDER_THEME=1 -DUSE_NSS=1 -DUSE_X11=1 -DOS_CHROMEOS=1 -DUSE_XI2_MT=2 -DFILE_MANAGER_EXTENSION=1 -DIMAGE_LOADER_EXTENSION=1 -DENABLE_REMOTING=1 -DENABLE_WEBRTC=1 -DUSE_PROPRIETARY_CODECS -DENABLE_PEPPER_CDMS -DENABLE_CONFIGURATION_POLICY -DENABLE_INPUT_SPEECH -DENABLE_NOTIFICATIONS -DENABLE_HIDPI=1 -DUSE_UDEV -DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC -DENABLE_EGLIMAGE=1 -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_AUTOMATION=1 -DENABLE_GOOGLE_NOW=1 -DCLD_VERSION=2 -DENABLE_FULL_PRINTING=1 -DENABLE_PRINTING=1 -DENABLE_SPELLCHECK=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_MANAGED_USERS=1 -DENABLE_MDNS=1 -DUSE_SYMBOLIZE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DBASE_IMPLEMENTATION -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -D_FORTIFY_SOURCE=2 -I../../.. -Werror -pthread -fno-exceptions -fno-strict-aliasing -Wall -Wno-unused-parameter -Wno-missing-field-initializers -fvisibility=hidden -pipe -fPIC -Wno-unused-local-typedefs -Wno-write-strings -pthread -I/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/include/glib-2.0 -I/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib/glib-2.0/include -march=armv7-a -mtune=cortex-a8 -mfpu=neon -mfloat-abi=hard -mthumb --sysroot=/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz -O2 -fno-ident -fdata-sections -ffunction-sections -funwind-tables -fno-rtti -fno-threadsafe-statics -fvisibility-inlines-hidden -Wsign-compare -Wno-abi  -c ../../../base/x11/edid_parser_x11.cc -o obj/base/x11/base.edid_parser_x11.o


vs a file in re2:
/usr/local/google/home/piman/goma-cros/armv7a-cros-linux-gnueabi-g++ -B/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+target_toolchain/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.22-gold -MMD -MF obj/third_party/re2/re2/re2.filtered_re2.o.d -DCHROMIUM_BUILD -DTOOLKIT_VIEWS=1 -DUSE_LIBJPEG_TURBO=1 -DENABLE_ONE_CLICK_SIGNIN -DENABLE_REMOTING=1 -DENABLE_WEBRTC=1 -DENABLE_CONFIGURATION_POLICY -DENABLE_INPUT_SPEECH -DENABLE_NOTIFICATIONS -DENABLE_EGLIMAGE=1 -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PLUGIN_INSTALLATION=1 -DENABLE_PLUGINS=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_AUTOMATION=1 -DENABLE_GOOGLE_NOW=1 -DENABLE_PRINTING=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_MESSAGE_CENTER=1 -DENABLE_SETTINGS_APP=1 -DENABLE_MANAGED_USERS=1 -D_FILE_OFFSET_BITS=64 -I../../.. -I../../../out/Default.Release/gen -I../../../third_party/re2 -fno-strict-aliasing -fvisibility=hidden -fstack-protector --param=ssp-buffer-size=4 -march=armv7-a -mfpu=neon -mfloat-abi=softfp -mtune=cortex-a8 -mthumb -funwind-tables -fPIC -pipe -pthread -Wendif-labels -Wno-missing-field-initializers -Wno-unused-parameter -Wno-write-strings -O2 -g2 -fno-exceptions -fno-threadsafe-statics -fvisibility-inlines-hidden -fno-rtti  -c ../../../third_party/re2/re2/filtered_re2.cc -o obj/third_party/re2/re2/re2.filtered_re2.o

So I worry that there's more than just softfp vs hard.

Brett Wilson

unread,
Jan 17, 2014, 7:30:30 PM1/17/14
to Antoine Labour, Haixia Shi, James Robinson, Ami Fischman, Chromium-dev
The current build is defined to be the flags necessary to get re2 to
compile. Most of these flags should be modularized instead of being
set globally. So re2 should not be getting things like
"use_libjpeg_turbo". These will need to be filled out as we get to
targets that require them.

Brett

Antoine Labour

unread,
Jan 17, 2014, 7:36:07 PM1/17/14
to Brett Wilson, Haixia Shi, James Robinson, Ami Fischman, Chromium-dev
Flags that are removed:
-DBASE_IMPLEMENTATION
-DCLD_VERSION=2
-DDYNAMIC_ANNOTATIONS_ENABLED=0
-DENABLE_FULL_PRINTING=1
-DENABLE_HIDPI=1
-DENABLE_MDNS=1
-DENABLE_PEPPER_CDMS
-DENABLE_RLZ
-DENABLE_SPELLCHECK=1
-DFILE_MANAGER_EXTENSION=1
-D_FORTIFY_SOURCE=2
-DGOOGLE_CHROME_BUILD
-DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC
-DIMAGE_LOADER_EXTENSION=1
-DNDEBUG
-DNVALGRIND
-DOS_CHROMEOS=1
-D__STDC_CONSTANT_MACROS
-D__STDC_FORMAT_MACROS
-DUI_COMPOSITOR_IMAGE_TRANSPORT
-DUSE_ASH=1
-DUSE_AURA=1
-DUSE_CAIRO=1
-DUSE_CRAS=1
-DUSE_DEFAULT_RENDER_THEME=1
-DUSE_GLIB=1
-DUSE_NSS=1
-DUSE_PROPRIETARY_CODECS
-DUSE_SYMBOLIZE
-DUSE_UDEV
-DUSE_X11=1
-DUSE_XI2_MT=2
-DV8_DEPRECATION_WARNINGS
-fdata-sections
-ffunction-sections
-fno-ident
-I/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/include/glib-2.0
-I/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz/usr/lib/glib-2.0/include
-mfloat-abi=hard
-pthread
--sysroot=/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz
-Wall
-Werror
-Wno-abi
-Wno-unused-local-typedefs
-Wsign-compare

Flags that are added:
-DCHROMIUM_BUILD
-DENABLE_MESSAGE_CENTER=1
-DENABLE_ONE_CLICK_SIGNIN
-DENABLE_PLUGIN_INSTALLATION=1
-DENABLE_SETTINGS_APP=1
-DUSE_LIBJPEG_TURBO=1
-fstack-protector
-g2
-I../../../out/Default.Release/gen
-I../../../third_party/re2
-mfloat-abi=softfp
--param=ssp-buffer-size=4
-Wendif-labels


Now, obviously, some only apply to base, and some probably are legitimately different in re, but the list looks like a lot more than this. -DNDEBUG, the -f and -W flags seem pretty important generally. We seem to be dropping --sysroot too.

Antoine

Antoine Labour

unread,
Jan 17, 2014, 7:52:22 PM1/17/14
to Brett Wilson, Haixia Shi, James Robinson, Ami Fischman, Chromium-dev
On Fri, Jan 17, 2014 at 4:30 PM, Brett Wilson <bre...@chromium.org> wrote:
Perhaps this is a more apples-to-apples comparison, comparing the same file before and after the change:

Flags removed:
-DCLD_VERSION=2
-DDYNAMIC_ANNOTATIONS_ENABLED=0
-DENABLE_FULL_PRINTING=1
-DENABLE_HIDPI=1
-DENABLE_MDNS=1
-DENABLE_PEPPER_CDMS
-DENABLE_RLZ
-DENABLE_SPELLCHECK=1
-DFILE_MANAGER_EXTENSION=1
-DGOOGLE_CHROME_BUILD
-DICU_UTIL_DATA_IMPL=ICU_UTIL_DATA_STATIC
-DIMAGE_LOADER_EXTENSION=1
-DNDEBUG
-DNVALGRIND
-DOS_CHROMEOS=1
-DUI_COMPOSITOR_IMAGE_TRANSPORT
-DUSE_ASH=1
-DUSE_AURA=1
-DUSE_CAIRO=1
-DUSE_CRAS=1
-DUSE_DEFAULT_RENDER_THEME=1
-DUSE_GLIB=1
-DUSE_NSS=1
-DUSE_PROPRIETARY_CODECS
-DUSE_UDEV
-DUSE_X11=1
-DUSE_XI2_MT=2
-DV8_DEPRECATION_WARNINGS
-fdata-sections
-ffunction-sections
-fno-ident
-mfloat-abi=hard
--sysroot=/mnt/d1/home/piman/work/chrome/.cros_cache/chrome-sdk/tarballs/daisy+5111.0.0+sysroot_chromeos-base_chromeos-chrome.tar.xz
-Wno-abi
-Wno-deprecated
-Wno-format
-Wno-unused-local-typedefs
-Wno-unused-result

Flags added:
-DCHROMIUM_BUILD
-DENABLE_MESSAGE_CENTER=1
-DENABLE_ONE_CLICK_SIGNIN
-DENABLE_PLUGIN_INSTALLATION=1
-DENABLE_SETTINGS_APP=1
-DUSE_LIBJPEG_TURBO=1
-fstack-protector
-g2
-I../../../out/Default.Release/gen
-mfloat-abi=softfp
--param=ssp-buffer-size=4
-Wendif-labels
-Wno-write-strings


Antoine

Nico Weber

unread,
Jan 17, 2014, 8:04:02 PM1/17/14
to Antoine Labour, Brett Wilson, Haixia Shi, James Robinson, Ami Fischman, Chromium-dev
This broke buildin on OS X for me: http://crbug.com/335819


--

Sam Clegg

unread,
Jan 17, 2014, 8:29:24 PM1/17/14
to pi...@google.com, Haixia Shi, James Robinson, Ami Fischman, Brett Wilson, Chromium-dev
Looks like the linux/ARM cross builder broke in the same way:

http://build.chromium.org/p/chromium.fyi/waterfall?show=Linux%20ARM%20Cross-Compile

/mnt/data/b/build/slave/Linux_ARM_Cross-Compile/build/src/third_party/gold/gold64:error:
obj/out/gn_gyp/third_party/re2/libre2.a(obj/out/gn_gyp/third_party/re2/../../../../third_party/re2/re2/re2.re2.o)
uses VFP register arguments, output does not

Brett Wilson

unread,
Jan 17, 2014, 8:34:57 PM1/17/14
to Sam Clegg, Antoine Labour, Haixia Shi, James Robinson, Ami Fischman, Chromium-dev
I just CQ+ed this change:
https://codereview.chromium.org/141143008/

If you think it looks right and want to take responsibility for
landing it, you can feel free to manually land it sooner.

Brett

James Robinson

unread,
Jan 19, 2014, 10:08:31 PM1/19/14
to Brett Wilson, Sam Clegg, Antoine Labour, Haixia Shi, Ami Fischman, Chromium-dev
The GN build now supports using an output directory other than 'out' either by specifying -Goutput_dir=... to gyp_chromium or setting the appropriate GYP_GENERATOR_FLAGS variable.  To support this, dependencies from gyp files to gn-generated gyp files must be specified using the gyp_output_dir gyp variable and not just 'out'.  For example, the path to the re2 target from the root of the repo is:

<(gyp_output_dir)/gn_gyp/third_party/re2/re2.gyp:re2

This variable is set by gyp_chromium itself. Please let me know if anything seems busted.

- James

Sergey Matveev

unread,
Jan 21, 2014, 7:22:47 AM1/21/14
to bre...@chromium.org, Chromium-dev
This change broke one of existing GYP flags somewhat. Setting msan=1 used to implicitly set clang=1, but since r245563 runhooks fails if msan=1 is specified without clang=1:

AssertionError: make_global_settings needs to be the same for all targets. [] vs. [['CC', 'third_party/llvm-build/Release+Asserts/bin/clang'], ['CXX', 'third_party/llvm-build/Release+Asserts/bin/clang++'], ['CC.host', '$(CC)'], ['CXX.host', '$(CXX)']]

This is weird because other flags with similar behavior, like asan=1 or tsan=1, are unaffected.


On Sat, Jan 18, 2014 at 2:33 AM, Brett Wilson <bre...@chromium.org> wrote:
I just converted the canonical version of the third_party/re2 target
to be GN and it seems to be sticking this time.

This means that when you run GYP, it will first run GN which will
convert src/third_party/re2/BUILD.gn into
out/gn_gyp/third_party/re2/re2.gyp. If you add new dependencies on
this target, you'll need to code the location of this GYP file:
http://src.chromium.org/viewvc/chrome/trunk/src/chrome/chrome_browser.gypi?r1=245563&r2=245562&pathrev=245563

As we add more stuff, more GYP files will appear in the out/gn_gyp
mirror tree, so please be aware that you may need to look there for
some targets. I don't want to write generated files to the main tree.
We can consider adding automatic fallback to this mirror tree to GYP
if it proves too annoying.

Recall there is documentation here:
https://code.google.com/p/chromium/wiki/gn including a quick start for
making a hello world target. I'll be in-and-out of the office for the
next few weeks but will be available to answer email if you have any
questions and will be working on fixing some of the issues.

--

This means you can now convert targets to GN without having to
maintain the GYP build in parallel. Currently, the best thing is to
find a static_library leaf-node (like in third_party).

Stuff that doesn't work now:

- Things that say 'toolsets': ['host'] in the target. There may or may
not be a "target" in that list as well. I haven't hooked up the "host"
cross-compile on Linux yet.

- Currently, only static libraries can be converted because there is
some linker stuff for Android that I still haven't hooked up for
shared libraries and executables. I hope to get this done soon.

- Running scripts aren't supported in the GYP output mode yet. So
don't try to convert any GYP actions.

--

There are some known issues. Since this is using the binary much more
than previously, there are some bugs. I know about a lock assertion on
Windows and a double-free on Linux that happen from time to time. They
generally will print a stack trace as part of running GYP, in a way
that is confusing where the error is actually coming from (due to
output buffering issues). I'll try to get these figured out soon.

In the meantime, if you see a stack trace and failure when running
hooks on your local machine or a buildbot, just try again.

There is some warnings spew from the re2. Feel free to look at which
warnings need to be disabled!

If you see an error about use_aura, being doubly-set (independent of
today's patch, this went in last week), remove use_aura from your
GYP_DEFINES. It's now redundant on a Windows build which is what this
error is telling you. I have a patch that removes the warning in this
case since it doesn't seem to be helping, but I haven't gotten the new
binary pushed yet.

Brett

Nico Weber

unread,
Jan 21, 2014, 1:25:38 PM1/21/14
to eart...@chromium.org, Brett Wilson, Chromium-dev
build/gyp_chromium has a remap list for stuff, it includes these 3:

      ('asan', '1', 'is_asan=true'),
      ('lsan', '1', 'is_lsan=true'),
      ('tsan', '1', 'is_tsan=true'),

but nothing for msan. You probably want to add that (send the CL to Brett or me).

This duplication is somewhat unfortunate; we'll see how annoying it is in practice over the coming weeks.

Nico Weber

unread,
Jan 21, 2014, 3:27:34 PM1/21/14
to Thomas Sepez, eart...@chromium.org, Brett Wilson, Chromium-dev
Please file a bug, cc Brett and me.


On Tue, Jan 21, 2014 at 12:26 PM, Thomas Sepez <tse...@google.com> wrote:
The following command has stopped working for me today with the same make_global_settings error:

GYP_DEFINES='asan=1 linux_use_tcmalloc=0 enable_ipc_fuzzer=1 release_extra_cflags="-g -O1 -fno-inline-functions -
fno-inline"' GYP_GENERATOR_FLAGS="output_dir=out_asan" gclient runhooks

Suggestions?

Sergey Matveev

unread,
Jan 21, 2014, 5:14:27 PM1/21/14
to Thomas Sepez, Nico Weber, Brett Wilson, Chromium-dev
Argh - wrong email address.


On Wed, Jan 22, 2014 at 2:13 AM, Sergey Matveev <eart...@google.com> wrote:
That's exactly the same issue. Just add clang=1 to GYP_DEFINES for a short term fix.



On Wed, Jan 22, 2014 at 12:26 AM, Thomas Sepez <tse...@google.com> wrote:
The following command has stopped working for me today with the same make_global_settings error:

GYP_DEFINES='asan=1 linux_use_tcmalloc=0 enable_ipc_fuzzer=1 release_extra_cflags="-g -O1 -fno-inline-functions -
fno-inline"' GYP_GENERATOR_FLAGS="output_dir=out_asan" gclient runhooks

Suggestions?
On Tue, Jan 21, 2014 at 10:25 AM, Nico Weber <tha...@chromium.org> wrote:

Nico Weber

unread,
Jan 21, 2014, 5:17:14 PM1/21/14
to Sergey Matveev, Thomas Sepez, Brett Wilson, Chromium-dev
On Tue, Jan 21, 2014 at 2:13 PM, Sergey Matveev <eart...@google.com> wrote:
That's exactly the same issue. Just add clang=1 to GYP_DEFINES for a short term fix.

Please just fix it as a short-term fix. It's like 3 lines.
 



On Wed, Jan 22, 2014 at 12:26 AM, Thomas Sepez <tse...@google.com> wrote:
The following command has stopped working for me today with the same make_global_settings error:

GYP_DEFINES='asan=1 linux_use_tcmalloc=0 enable_ipc_fuzzer=1 release_extra_cflags="-g -O1 -fno-inline-functions -
fno-inline"' GYP_GENERATOR_FLAGS="output_dir=out_asan" gclient runhooks

Suggestions?
On Tue, Jan 21, 2014 at 10:25 AM, Nico Weber <tha...@chromium.org> wrote:

Nico Weber

unread,
Jan 21, 2014, 7:51:11 PM1/21/14
to Sergey Matveev, Thomas Sepez, Brett Wilson, Chromium-dev
On Tue, Jan 21, 2014 at 2:17 PM, Nico Weber <tha...@chromium.org> wrote:
On Tue, Jan 21, 2014 at 2:13 PM, Sergey Matveev <eart...@google.com> wrote:
That's exactly the same issue. Just add clang=1 to GYP_DEFINES for a short term fix.

Please just fix it as a short-term fix. It's like 3 lines.

Done in r246188.

Newton Allen

unread,
Jan 21, 2014, 10:34:08 PM1/21/14
to James Robinson, Brett Wilson, Sam Clegg, Antoine Labour, Haixia Shi, Ami Fischman, Chromium-dev
Sweet. Thanks for the change, James!

I've hit some busted behavior though. Specifying an absolute path for -Goutput_dir=... causes gn to crash. Stack trace below.

clankium:/$ build/gyp_chromium -Goutput_dir=/usr/local/google/code/clankium/src/out
Generating gyp files from GN...
ERROR at //device/usb/BUILD.gn:6:17: File not inside output directory.
generated_ids = "$target_gen_dir/usb_ids_gen.cc"
                ^-------------------------------
The given file should be in the output directory. Normally you would specify
"$target_output_dir/foo" or "$target_gen_dir/foo". I interpreted this as
"//usr/local/google/code/clankium/src/out/gn_build.Release/gen/device/usb/usb_ids_gen.cc".
[0121/192419:FATAL:file_path.cc(485)] Check failed: !IsPathAbsolute(*appended).
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4fe2be]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4b7fc0]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4b4d47]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4b4f69]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x45c788]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x470b12]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4231b1]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x41f9c6]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4411e0]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x443325]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x49c612]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4412b0]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4424d2]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x443050]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4424d2]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x443050]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4424d2]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x41f789]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4411e0]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4424d2]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x442618]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4367cb]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x432914]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x432d41]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4e98a7]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4ea6b7]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4ebd19]
/usr/local/google/code/clankium/src/tools/gn/bin/linux/gn() [0x4e605e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a) [0x7ffd01e3be9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffd01b683fd]
 

Brett Wilson

unread,
Jan 22, 2014, 2:55:52 AM1/22/14
to Newton Allen, James Robinson, Sam Clegg, Antoine Labour, Haixia Shi, Ami Fischman, Chromium-dev
There are a couple of issues. GN needs to support absolute output
directories located anywhere on the computer. This hasn't been
implemented yet and it asserts. Independent of that is whether this
can be supported in the hybrid GN/GYP build. Given the way that we
express the generated .gyp file locations, for example:

'../<(gyp_output_dir)/gn_gyp/third_party/re2/re2.gyp:re2',

it's not particularly convenient to do. I think you should be able to
just use an absolute directory here for now.

Brett

Brett Wilson

unread,
Jan 22, 2014, 4:28:24 AM1/22/14
to Chromium-dev
I just reverted this mostly due to the completely nonexistent iOS
support. Will try to land again in a few days with more stuff fixed.

Brett

Sergey Matveev

unread,
Jan 22, 2014, 5:59:14 AM1/22/14
to Nico Weber, Thomas Sepez, Brett Wilson, Chromium-dev
Thanks. It wasn't exactly business hours here when I replied.

Daniel Bratell

unread,
Jan 28, 2014, 11:55:42 AM1/28/14
to Chromium-dev, Brett Wilson
On Fri, 17 Jan 2014 23:33:34 +0100, Brett Wilson <bre...@chromium.org>
wrote:

> There are some known issues.

[snip]

> I know about a lock assertion on
> Windows and a double-free on Linux that happen from time to time.

I assume that is this:

*** glibc detected ***
/home/buildbot/buildbot/slave/workdir/repos/RlDS/chromium/src/tools/gn/bin/linux/gn:
double free or corruption (fasttop): 0x00007f571c017910 ***

I see it happening at our build servers (building a 10 day old version of
Chromium: 1788.0) and they are unfortunately not intelligent enough to
just try again.

Did you figure out what the problem is or do you have another suggestion
for how to avoid crashing builds? 10 days ago, maybe it's possible to just
comment out the gn call?

/Daniel

Nico Weber

unread,
Jan 28, 2014, 11:57:14 AM1/28/14
to bratell at Opera, Chromium-dev, Brett Wilson
This is https://code.google.com/p/chromium/issues/detail?id=335587 . I think we stopped seeing it on our bots – if you can repro, can you try to get a stack?
 


/Daniel

Daniel Bratell

unread,
Jan 28, 2014, 12:07:24 PM1/28/14
to Nico Weber, Chromium-dev, Brett Wilson
On Tue, 28 Jan 2014 17:57:14 +0100, Nico Weber <tha...@chromium.org> wrote:

> On Tue, Jan 28, 2014 at 8:55 AM, Daniel Bratell <bra...@opera.com>
> wrote:
>
>> *** glibc detected ***
>> /home/buildbot/buildbot/slave/workdir/repos/RlDS/chromium/src/tools/gn/bin/>>linux/gn:
>> double free or corruption (fasttop): 0x00007f571c017910 ***
>>
>> I see it happening at our build servers (building a 10 day old version
>> of Chromium: 1788.0) and they >>are unfortunately not intelligent
>> enough to just try again.
>>
>> Did you figure out what the problem is or do you have another
>> suggestion for how to avoid crashing >>builds? 10 days ago, maybe it's
>> possible to just comment out the gn call?
>
> This is https://code.google.com/p/chromium/issues/detail?id=335587 . I
> think we stopped seeing it on >our bots – if you can repro, can you try
> to get a stack?

It reports a stack without symbols (just addresses) but that one is in the
bug report. Otherwise it's not that easy. I have so far only seen it on
the build machines.

Is there a way to make it run a debug build of gn instead of the release
build? Considering the error rate so far I think it should trigger a few
times if I start ten builds.

Alternatively, if I make it use the gn from trunk today, do you think that
could fix the problems without causing other problems?

/Daniel

Nico Weber

unread,
Jan 28, 2014, 12:08:47 PM1/28/14
to Daniel Bratell, Chromium-dev, Brett Wilson
On Tue, Jan 28, 2014 at 9:07 AM, Daniel Bratell <bra...@opera.com> wrote:
On Tue, 28 Jan 2014 17:57:14 +0100, Nico Weber <tha...@chromium.org> wrote:

On Tue, Jan 28, 2014 at 8:55 AM, Daniel Bratell <bra...@opera.com> wrote:

*** glibc detected *** /home/buildbot/buildbot/slave/workdir/repos/RlDS/chromium/src/tools/gn/bin/>>linux/gn: double free or corruption (fasttop): 0x00007f571c017910 ***


I see it happening at our build servers (building a 10 day old version of Chromium: 1788.0) and they >>are unfortunately not intelligent enough to just try again.

Did you figure out what the problem is or do you have another suggestion for how to avoid crashing >>builds? 10 days ago, maybe it's possible to just comment out the gn call?

This is https://code.google.com/p/chromium/issues/detail?id=335587 . I think we stopped seeing it on >our bots – if you can repro, can you try to get a stack?

It reports a stack without symbols (just addresses) but that one is in the bug report. Otherwise it's not that easy. I have so far only seen it on the build machines.

Is there a way to make it run a debug build of gn instead of the release build? Considering the error rate so far I think it should trigger a few times if I start ten builds.

Sure: `ninja -C out/Debug gn`.

Nico Weber

unread,
Jan 28, 2014, 12:09:51 PM1/28/14