Message:
Evan: please review.
Antoine: FYI.
I replaced the specialness of _host with _target, b/c the _host toolchain is
usually the vanilla gcc/g++, so it looks better to leave those variables
alone
(plus, it mirrors make.py). Also added an AR to the family b/c it differs
between host & target. Finally, I made generator_supports_multiple_toolsets
depend on the presence of any of the $FOO_target variables, b/c I don't
know how
to fix the double-building problem for realz.
FTR, I got my x64 z600 to build a chrome binary that ran correctly on my
cros/tegra2 ARM notebook with the following. I'm not particularly pleased
about
the amount of customization needed, but it mirrors what's necessary for
make, so
I can live with it ok.
BASE="armv7a-cros-linux-gnueabi"
GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/"
SYSROOT="--sysroot=/build/BOARD"
ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16 -mfloat-abi=softfp"
BACKTRACE="-funwind-tables -rdynamic"
export GYP_GENERATORS=ninja
export AR_target=$BASE-ar
export CC_target="$BASE-gcc $SYSROOT $BACKTRACE $ARMIFY"
export CXX_target="$BASE-g++ $GOLDIFY $SYSROOT $GOLDIFY $BACKTRACE $ARMIFY"
./build/gyp_chromium && ../goma-ninja chrome
Description:
Enable cross-compilation with ninja.
Please review this at https://chromiumcodereview.appspot.com/9417030/
SVN Base: http://git.chromium.org/external/gyp.git@master
Affected files:
M pylib/gyp/generator/ninja.py
Index: pylib/gyp/generator/ninja.py
diff --git a/pylib/gyp/generator/ninja.py b/pylib/gyp/generator/ninja.py
index
4f5b7d94ca0c7f6bd3333423b27d9c58b159a3aa..c02ece86883615f83c36225ce7fef7d611c4e46f
100644
--- a/pylib/gyp/generator/ninja.py
+++ b/pylib/gyp/generator/ninja.py
@@ -42,11 +42,11 @@ generator_default_variables = {
'RULE_INPUT_NAME': '${name}',
}
-# TODO: enable cross compiling once we figure out:
-# - how to not build extra host objects in the non-cross-compile case.
-# - how to decide what the host compiler is (should not just be $cc).
-# - need ld_host as well.
-generator_supports_multiple_toolsets = False
+# TODO: figure out how to not build extra host objects in the
non-cross-compile
+# case when this is enabled, and enable unconditionally.
+generator_supports_multiple_toolsets = (
+ os.environ.get('AR_target') or os.environ.get('CC_target') or
+ os.environ.get('CXX_target'))
def StripPrefix(arg, prefix):
@@ -569,9 +569,11 @@ class NinjaWriter:
def WriteSources(self, config_name, config, sources, predepends,
precompiled_header):
"""Write build rules to compile all of |sources|."""
- if self.toolset == 'host':
- self.ninja.variable('cc', '$cc_host')
- self.ninja.variable('cxx', '$cxx_host')
+ if self.toolset == 'target':
+ self.ninja.variable('ar', '$ar_target')
+ self.ninja.variable('cc', '$cc_target')
+ self.ninja.variable('cxx', '$cxx_target')
+ self.ninja.variable('ld', '$ld_target')
if self.flavor == 'mac':
cflags = self.xcode_settings.GetCflags(config_name)
@@ -1041,14 +1043,22 @@ def GenerateOutputForConfig(target_list,
target_dicts, data, params,
flock = 'flock'
if flavor == 'mac':
flock = './gyp-mac-tool flock'
+ master_ninja.variable('ar', os.environ.get('AR', 'ar'))
master_ninja.variable('cc', os.environ.get('CC', cc))
master_ninja.variable('cxx', os.environ.get('CXX', cxx))
if flavor == 'win':
master_ninja.variable('ld', 'link')
else:
master_ninja.variable('ld', flock + ' linker.lock $cxx')
- master_ninja.variable('cc_host', '$cc')
- master_ninja.variable('cxx_host', '$cxx')
+
+ master_ninja.variable('ar_target', os.environ.get('AR_target', 'ar'))
+ master_ninja.variable('cc_target', os.environ.get('CC_target', cc))
+ master_ninja.variable('cxx_target', os.environ.get('CXX_target', cxx))
+ if flavor == 'win':
+ master_ninja.variable('ld_target', 'link')
+ else:
+ master_ninja.variable('ld_target', flock + ' linker.lock $cxx_target')
+
if flavor == 'mac':
master_ninja.variable('mac_tool', os.path.join('.', 'gyp-mac-tool'))
master_ninja.newline()
@@ -1090,7 +1100,7 @@ def GenerateOutputForConfig(target_list,
target_dicts, data, params,
master_ninja.rule(
'alink',
description='AR $out',
- command='rm -f $out && ar rcsT $out $in')
+ command='rm -f $out && $ar rcsT $out $in')
master_ninja.rule(
'solink',
description='SOLINK $out',
> I replaced the specialness of _host with _target, b/c the _host toolchain
> is
> usually the vanilla gcc/g++, so it looks better to leave those variables
> alone
> (plus, it mirrors make.py). Also added an AR to the family b/c it differs
> between host & target. Finally, I made
> generator_supports_multiple_toolsets
> depend on the presence of any of the $FOO_target variables, b/c I don't
> know
how
> to fix the double-building problem for realz.
Can you put some of these comments in the review description?
BTW, it appears GypPathToUniqueOutput also makes "target" be the special
case.
> FTR, I got my x64 z600 to build a chrome binary that ran correctly on my
> cros/tegra2 ARM notebook with the following. I'm not particularly pleased
about
> the amount of customization needed, but it mirrors what's necessary for
> make,
so
> I can live with it ok.
> BASE="armv7a-cros-linux-gnueabi"
GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/"
> SYSROOT="--sysroot=/build/BOARD"
> ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16
> -mfloat-abi=softfp"
> BACKTRACE="-funwind-tables -rdynamic"
FWIW, I suspect some of that belongs in common.gypi.
> export GYP_GENERATORS=ninja
> export AR_target=$BASE-ar
> export CC_target="$BASE-gcc $SYSROOT $BACKTRACE $ARMIFY"
> export CXX_target="$BASE-g++ $GOLDIFY $SYSROOT $GOLDIFY $BACKTRACE
> $ARMIFY"
> ./build/gyp_chromium && ../goma-ninja chrome
Done.
> > ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16
> -mfloat-abi=softfp"
> > BACKTRACE="-funwind-tables -rdynamic"
> FWIW, I suspect some of that belongs in common.gypi.
Yeah; there's probably some overlap and some missing. I'll look into it.
Thanks. Are you a gyp project member/ can you land this for me?
-a
LGTM
https://chromiumcodereview.appspot.com/9417030/
Reviewers: Evan Martin,
Message:
Evan: please review.
Antoine: FYI.
I replaced the specialness of _host with _target, b/c the _host toolchain is
usually the vanilla gcc/g++, so it looks better to leave those variables alone
(plus, it mirrors make.py). Also added an AR to the family b/c it differs
between host & target. Finally, I made generator_supports_multiple_toolsets
depend on the presence of any of the $FOO_target variables, b/c I don't know how
to fix the double-building problem for realz.
FTR, I got my x64 z600 to build a chrome binary that ran correctly on my
cros/tegra2 ARM notebook with the following. I'm not particularly pleased about
the amount of customization needed, but it mirrors what's necessary for make, so
I can live with it ok.
BASE="armv7a-cros-linux-gnueabi"
GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/"
SYSROOT="--sysroot=/build/BOARD"
ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16 -mfloat-abi=softfp"
BACKTRACE="-funwind-tables -rdynamic"
export GYP_GENERATORS=ninja
export AR_target=$BASE-ar
export CC_target="$BASE-gcc $SYSROOT $BACKTRACE $ARMIFY"
export CXX_target="$BASE-g++ $GOLDIFY $SYSROOT $GOLDIFY $BACKTRACE $ARMIFY"
In the make world, we expect $CC to be the target compiler. With this, $CC is taken for the host compiler... That seems inconsistent.
In the make world, we expect $CC to be the target compiler. With this, $CC is taken for the host compiler... That seems inconsistent.Paraphrasing from the other thread:Can you be more explicit (code pointers)? My statements were based on make.py's use of {CC,CXX,AR,{C,CXX,LD,AR}FLAGS,LINK}.target here:
In the other thread I said I didn't really care which way they went, as long as {ninja,make}.py agreed, but actually I do.The way I implemented ninja.py (and believe make.py works as well), the same CC/CXX works for both non-cross-compiles & cross-compiles (where the latter also require setting the .target variants to something else).The way you propose requires the cross-compile & non-cross-compile cases to use different settings for CC/CXX.I think the former is uniformly better than the latter b/c it supports more scenarios with less configuration.
Cheers,-a
I'm not sure how it can be more explicit. "CC.target ?= $(CC)", that means $CC is what's used for the target compiler.
All systems I know use $CC to mean the target compiler (not host). Most projects don't even need a host compiler.When compiling something for a different architecture, I'm expecting to provide a different compiler, so it makes sense to me to set $CC differently, I'm not sure why it's a problem (since in your solution you also need different settings, this time for $CC_target instead).
BASE="armv7a-cros-linux-gnueabi"GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/"SYSROOT="--sysroot=/build/tegra2_kaen"
ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16 -mfloat-abi=softfp"BACKTRACE="-funwind-tables -rdynamic"
PATH=~/goma:$PATHmake -j500 \CC.target="$BASE-gcc" CXX.target="$BASE-g++" \LINK.target="$BASE-g++ $GOLDIFY" AR.target=$BASE-ar \CFLAGS.target="$SYSROOT $ARMIFY $BACKTRACE" \CXXFLAGS.target="$SYSROOT $ARMIFY $BACKTRACE" \LDFLAGS.target="$SYSROOT $BACKTRACE" \"$@"
export ARMV7BASE="armv7a-cros-linux-gnueabi"export GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/ -Wl,--threads -Wl,--thread-count=4"export SYSROOTFLAG="--sysroot=$SYSROOT"export ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16 -mfloat-abi=softfp"export BACKTRACE="-funwind-tables -rdynamic"export GYP_GENERATORS=ninjaexport AR_target=$ARMV7BASE-arexport CC_target="$ARMV7BASE-gcc $SYSROOTFLAG $BACKTRACE $ARMIFY"export CXX_target="$ARMV7BASE-g++ $SYSROOTFLAG $GOLDIFY $BACKTRACE $ARMIFY"./build/gyp_chromium && ../goma-ninja chrome
I'm not sure how it can be more explicit. "CC.target ?= $(CC)", that means $CC is what's used for the target compiler.No, it only means $CC is used as the target compiler when not cross-compiling (i.e. when host==target).
All systems I know use $CC to mean the target compiler (not host). Most projects don't even need a host compiler.When compiling something for a different architecture, I'm expecting to provide a different compiler, so it makes sense to me to set $CC differently, I'm not sure why it's a problem (since in your solution you also need different settings, this time for $CC_target instead).I think it's not possible we truly disagree about the desired interface for developers, and that we must be miscommunicating. Let me take a different tack and see if it makes sense to you; the important bit is in bold below.In the make world I used to do this to x-compile chrome for ARM on my z600:BASE="armv7a-cros-linux-gnueabi"GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/"SYSROOT="--sysroot=/build/tegra2_kaen"ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16 -mfloat-abi=softfp"BACKTRACE="-funwind-tables -rdynamic"PATH=~/goma:$PATHmake -j500 \CC.target="$BASE-gcc" CXX.target="$BASE-g++" \LINK.target="$BASE-g++ $GOLDIFY" AR.target=$BASE-ar \CFLAGS.target="$SYSROOT $ARMIFY $BACKTRACE" \CXXFLAGS.target="$SYSROOT $ARMIFY $BACKTRACE" \LDFLAGS.target="$SYSROOT $BACKTRACE" \"$@"
In the new ninja world I do this for the same build:export ARMV7BASE="armv7a-cros-linux-gnueabi"export GOLDIFY="-B/usr/x86_64-pc-linux-gnu/armv7a-cros-linux-gnueabi/binutils-bin/2.21-gold/ -Wl,--threads -Wl,--thread-count=4"export SYSROOTFLAG="--sysroot=$SYSROOT"export ARMIFY="-march=armv7-a -mtune=cortex-a8 -mfpu=vfpv3-d16 -mfloat-abi=softfp"export BACKTRACE="-funwind-tables -rdynamic"export GYP_GENERATORS=ninjaexport AR_target=$ARMV7BASE-arexport CC_target="$ARMV7BASE-gcc $SYSROOTFLAG $BACKTRACE $ARMIFY"export CXX_target="$ARMV7BASE-g++ $SYSROOTFLAG $GOLDIFY $BACKTRACE $ARMIFY"./build/gyp_chromium && ../goma-ninja chromeThe same ../goma-ninja script (which adds goma to $PATH and takes care of ninja's -C param) is used for both cross-compiling and non-cross-compiling using ninja.Note in both setups I needed to specify the cross-toolchain in .target or _target vars, whereas the host toolchain uses the unqualified defaults (bare, unqualified, gcc/g++ or clang/clang++).In neither (ninja nor make) setup is there a _host or .host variant.
IMO these snippets demonstrate that ninja.py implements the same host/target model as make.py.
If you still think ninja.py implements a different model from make.py can you give me the equivalent setups that show the difference?
Cheers,-a
I'm not sure how it can be more explicit. "CC.target ?= $(CC)", that means $CC is what's used for the target compiler.No, it only means $CC is used as the target compiler when not cross-compiling (i.e. when host==target).In make you're always effectively cross-compiling. You set $CC and you're done with it. You don't have to override CC.target explicitly. I don't. Chrome OS build scripts don't. The (older) ARM build instructions for make don't.
Personally I just export CC, CXX, etc. like the rest of my environment rather than overriding the make variables on the command line.
In neither (ninja nor make) setup is there a _host or .host variant.I don't set it up either.
In the make world, we expect $CC to be the target compiler. With this, $CC is taken for the host compiler... That seems inconsistent.
I'm not sure how it can be more explicit. "CC.target ?= $(CC)", that means $CC is what's used for the target compiler.No, it only means $CC is used as the target compiler when not cross-compiling (i.e. when host==target).In make you're always effectively cross-compiling. You set $CC and you're done with it. You don't have to override CC.target explicitly. I don't. Chrome OS build scripts don't. The (older) ARM build instructions for make don't.If you don't override CC.target and put the target compiler in CC, where do you specify the host compiler?
Personally I just export CC, CXX, etc. like the rest of my environment rather than overriding the make variables on the command line.Of course I do too, but you missed my point.You're telling me how you build for host==target.
I'm telling you how I build when host!=target.In neither (ninja nor make) setup is there a _host or .host variant.I don't set it up either.But up-thread you said:In the make world, we expect $CC to be the target compiler. With this, $CC is taken for the host compiler... That seems inconsistent.which I read as you lobbying for ninja.py to require setting CC_host instead of CC for the host compiler.
No, that's how I build for host!=target. Try it.
No, that's how I build for host!=target. Try it.Whoah! We're both right! ninja.py works like make.py! And also ninja.py does NOT work like make.py!Turns out make.py supports host!=target in two different ways:- {CC,CXX} set to cross-toolchain, {CC,CXX}.host default to gcc/g++ (your way, which is news to me as of this morning)- {CC,CXX}_target set to cross-toolchain, {CC,CXX}{,.host} default to gcc/g++ (my way; did you realize it supported both?)ninja.py implements my way, b/c I didn't know about make.py's support for your way.Perhaps a summary of the problem is that three variables are one too many for describing only two compilers.IIUC you want to:- Make CC/CXX always be the target compiler
- Drop the {_,.}target family of vars
- (continue to) Default the {_,.}host family to gcc/g++, and allow overriding it for crazier hosting setups.
I don't have a problem with this but I don't think it's worth doing just in ninja.py.
Are you proposing that make.py drop support for my way, too?
-a
- Drop the {_,.}target family of varsI don't really care - for me they're just internal variables. When I added support for cross-compilation to make.py, I didn't intend for them to be used externally, but if they're useful, go for it.
- (continue to) Default the {_,.}host family to gcc/g++, and allow overriding it for crazier hosting setups.We may want to allow overriding for building with clang. I think Nico needed that.Also, in the current version of ninja.py, the host compilers default to $CC not gcc, which would break in "my way".
Are you proposing that make.py drop support for my way, too?Not at all.