How to disable GCC parallel compilation of universal binaries?
Gabriele Greco
compiling quite heavy C++ sources and my machine has not enough memory
to keep TWO instances of GCC compiling them in memory at once..
I ran through configure/make command line, and I have configured with
CXXFLAGS and LDFLAGS:
-arch i386 -arch ppc
There is some other flag that let you tell GCC not to compile the two
archs at the same time?
Bye,
Gabry
vze3...@verizon.net
show Architectures near the top marked for both ppc & i386. Click on
Architectures and uncheck one of them in the panel that appears.
--jim
Michael Ash
There is no reason to compile for both architectures during development.
Since your development machine is i386, kill the -arch ppc. You can put it
back when you're ready to build for deployment and, I suppose, just suffer
through the build times in that case.
As an alternative, you can build once with -arch i386 and a second time
with -arch ppc, then use /usr/bin/lipo to stitch the two binaries
together.
--
Michael Ash
Rogue Amoeba Software
Gabriele Greco
> Click on your project and in the Build look in All settings. It should
> show Architectures near the top marked for both ppc & i386. Click on
> Architectures and uncheck one of them in the panel that appears.
No, I want to build BOTH architectures and generate a universal binary
BUT I want to compile the objects sequentially and not in parallel like
gcc behaviour is...
Anyway I don't use xcode, at least for this project, I've not enough
memory to run ALSO xcode (GCC on large sources allocate more than 400mb
per file, if ps reports are correct, and my machine became a swap box),
upgrade the memory on a mac mini (512mb) is not that easy and I cannot
risk to loose it at the moment :)
Bye,
Gabry
Patrick Machielse
> I want to build BOTH architectures and generate a universal binary
> BUT I want to compile the objects sequentially and not in parallel like
> gcc behaviour is...
>
> Anyway I don't use xcode, at least for this project, I've not enough
> memory to run ALSO xcode (GCC on large sources allocate more than 400mb
> per file, if ps reports are correct, and my machine became a swap box),
> upgrade the memory on a mac mini (512mb) is not that easy and I cannot
> risk to loose it at the moment :)
If you use make, you can specify the -j option. From the man page:
-j jobs
Specifies the number of jobs (commands) to run simultaneously. If
there is more than one -j option, the last one is effective. If
the -j option is given without an argument, make will not limit
the number of jobs that can run simultaneously.
I vagely remember hearing that the optimal value is 2 * # of processor
cores, but don't pin me down on it... In your case you could try 1.
patrick
Tom Harrington
Gabriele Greco <gabrie...@REMOVEUPPERCASETOMAILtin.it> wrote:
So you're using a Makefile? That's where you want to look. GCC doesn't
start itself up multiple times, but make may start multiple copies of
gcc if it's configured to do so and if the dependency chain allows it.
--
Tom "Tom" Harrington
MondoMouse makes your mouse mightier
See http://www.atomicbird.com/mondomouse/
Clark S. Cox III
OK, so you're calling gcc directly?
Then just split the archs up yourself, and lipo them together:
gcc -arch ppc <common args here> -O foo.o.ppc
gcc -arch i386 <common args here> -O foo.o.i386
lipo -create foo.o.ppc foo.o.i386 -output foo.o
--
Clark S. Cox III
clar...@gmail.com
Santa Claus
Michael Ash <mi...@mikeash.com> wrote:
you can also use xcode and only compile one architecture at a time and
then turn them both on and build without cleaning.
Gabriele Greco
> Then just split the archs up yourself, and lipo them together:
>
> gcc -arch ppc <common args here> -O foo.o.ppc
> gcc -arch i386 <common args here> -O foo.o.i386
> lipo -create foo.o.ppc foo.o.i386 -output foo.o
So I have to create two source trees because I pass through a configure
script (automake/autoconf).
The only thing I would like to know if it's possible to break parallel
compilation of GCC when you type:
g++ -o target -arch i386 -arch ppc foo.cpp
- I know I can build only an arch and then "lipo" them together, but
this is not an option because I'll need two source trees to do that.
It seems REALLY strange that GCC implemented such a complex system to
double his instances when you specify more than one arch and then wait
for both to complete before linking and does not give an option to do it
sequentially.
I'm not using -j feature of makefile, gcc compiles on two threads the
two architectures by himself, and I think that forcing this design by
default is quite broken also if with enough memory it may be useful.
I'd like to be able to do something like this:
g++ -o target -arch i386 -arch ppc foo.cpp -fno-parallel-compile
... but I've not find anywhere some documentation to obtain this behaviour.
With 512mb I obtain the following results:
1) compilation of a SINGLE source file with double arch:
15mins
2) compilation of the SAME source file with one single arch:
90 sec
This difference is because my machine became a swap box...
Bye,
Gabry
Patrick Machielse
> I'm not using -j feature of makefile, gcc compiles on two threads the
> two architectures by himself, and I think that forcing this design by
> default is quite broken also if with enough memory it may be useful.
Have you tried '-j 1'? According to the man page, when you *don't*
specify a value, make is free to create as many jobs as it sees fit.
My gut feeling is that this is a feature of the build system, not of the
compiler.
patrick
Clark S. Cox III
The OP's problem isn't one of *make* spawning multiple processes, so the
-j option won't help here. Make is simply calling a single instance of
gcc, it's *gcc* that is actually performing a pair of compiles and
lipo-ing them together under the hood.
To the OP:
I suppose you could write a small shell-script that called the two gcc
instances in sequence, followed by the lipo, and use *that* as your
compiler.
Patrick Machielse
> The OP's problem isn't one of *make* spawning multiple processes, so the
> -j option won't help here. Make is simply calling a single instance of
> gcc, it's *gcc* that is actually performing a pair of compiles and
> lipo-ing them together under the hood.
Yes, but as far as I can tell, the compiles don't always happen in
parallel. I never see more than 1 compile process at a time on my
(single core) machine.
On my sluggish PowerBook G4 / 512 MB, 7 file project:
- ppc only: 50 Sec.
- universal: 110 Sec.
GCC probably makes a different decision on a multi core system. So maybe
the solution is to disable one core ;-)
patrick
Michael Ash
>
>> Then just split the archs up yourself, and lipo them together:
>>
>> gcc -arch ppc <common args here> -O foo.o.ppc
>> gcc -arch i386 <common args here> -O foo.o.i386
>> lipo -create foo.o.ppc foo.o.i386 -output foo.o
>
> So I have to create two source trees because I pass through a configure
> script (automake/autoconf).
>
> The only thing I would like to know if it's possible to break parallel
> compilation of GCC when you type:
>
> g++ -o target -arch i386 -arch ppc foo.cpp
Again, why can't you just compile for one architecture? It's hard to
imagine a situation where you need to create a universal binary for every
single build.
> - I know I can build only an arch and then "lipo" them together, but
> this is not an option because I'll need two source trees to do that.
You certainly don't. Build once, move the resulting binary somewhere else,
build again, lipo them together. This is fairly trivial to automate.
Ben Artin
> In comp.sys.mac.programmer.help Gabriele Greco
> <gabrie...@removeuppercasetomailtin.it> wrote:
> > I know it's a useful feature since I have a core duo processor, but I'm
> > compiling quite heavy C++ sources and my machine has not enough memory
> > to keep TWO instances of GCC compiling them in memory at once..
>
> There is no reason to compile for both architectures during development.
> Since your development machine is i386, kill the -arch ppc. You can put it
> back when you're ready to build for deployment and, I suppose, just suffer
> through the build times in that case.
That's irrelevant because Xcode will just spin off two i386 compiles instead
when it sees 2 CPUs. The real question is how to tell Xcode how many CPUs to
use, and I don't know the answer to that :-(
Ben
--
If this message helped you, consider buying an item
from my wish list: <http://artins.org/ben/wishlist>
I changed my name: <http://periodic-kingdom.org/People/NameChange.php>
Michael Ash
> In article <11691241...@nfs-db1.segnet.com>,
> Michael Ash <mi...@mikeash.com> wrote:
>
>> In comp.sys.mac.programmer.help Gabriele Greco
>> <gabrie...@removeuppercasetomailtin.it> wrote:
>> > I know it's a useful feature since I have a core duo processor, but I'm
>> > compiling quite heavy C++ sources and my machine has not enough memory
>> > to keep TWO instances of GCC compiling them in memory at once..
>>
>> There is no reason to compile for both architectures during development.
>> Since your development machine is i386, kill the -arch ppc. You can put it
>> back when you're ready to build for deployment and, I suppose, just suffer
>> through the build times in that case.
>
> That's irrelevant because Xcode will just spin off two i386 compiles instead
> when it sees 2 CPUs. The real question is how to tell Xcode how many CPUs to
> use, and I don't know the answer to that :-(
*That* is irrelevant because he's not using Xcode.
But as to the point you raise, a bit of searching turns up
PBXNumberOfParallelBuildSubtasks. I have gobs of RAM and so I have never
actually tried it, however. Parallel builds are Good For You.
Gabriele Greco
> Again, why can't you just compile for one architecture? It's hard to
> imagine a situation where you need to create a universal binary for every
> single build.
Well the project is in release stage, not development, every build
potentially is a release build (I'm working on a patch).
Building everything from scratch needs 4 hours on my system.
Running "configure" every time I need to switch arch is not that
comfortable, I could do it with two source trees, but then I'll need
also to keep them updated both and do tons of cvs update/commit from one
tree to the other...
>> - I know I can build only an arch and then "lipo" them together, but
>> this is not an option because I'll need two source trees to do that.
>
> You certainly don't. Build once, move the resulting binary somewhere else,
> build again, lipo them together. This is fairly trivial to automate.
>
It's not trivial if you depend from the standard automake/autoconf build
system.
The best solution so far is to use a fake gcc script that runs two gccs
and lipo the objects together, I've already done something similar to
build an universal binary that works on 10.2 (I got the template from
the gcc-fat.sh available in the libsdl site).
Bye,
Gabry
Ben Artin
Michael Ash <mi...@mikeash.com> wrote:
> In comp.sys.mac.programmer.help Ben Artin <mac...@artins.org> wrote:
> > In article <11691241...@nfs-db1.segnet.com>, Michael Ash
> > <mi...@mikeash.com> wrote:
> >
> >> In comp.sys.mac.programmer.help Gabriele Greco
> >> <gabrie...@removeuppercasetomailtin.it> wrote:
> >> > I know it's a useful feature since I have a core duo processor, but I'm
> >> > compiling quite heavy C++ sources and my machine has not enough memory
> >> > to keep TWO instances of GCC compiling them in memory at once..
> >>
> >> There is no reason to compile for both architectures during development.
> >> Since your development machine is i386, kill the -arch ppc. You can put it
> >> back when you're ready to build for deployment and, I suppose, just suffer
> >> through the build times in that case.
> >
> > That's irrelevant because Xcode will just spin off two i386 compiles
> > instead when it sees 2 CPUs. The real question is how to tell Xcode how
> > many CPUs to use, and I don't know the answer to that :-(
>
> *That* is irrelevant because he's not using Xcode.
*THAT* is a good point, but then AFAIK gcc doesn't perform any parallelization
-- one invocation of gcc equals one process, no parallel builds. Even if gcc is
compiling for two archs from one invocation of gcc, I believe that still happens
on a single processor.
Michael Ash
> *THAT* is a good point, but then AFAIK gcc doesn't perform any parallelization
> -- one invocation of gcc equals one process, no parallel builds. Even if gcc is
> compiling for two archs from one invocation of gcc, I believe that still happens
> on a single processor.
I would have said the same thing before this thread, but I just tested it,
and passing two arch flags really does result in two invocations of the
compiler running simultaneously:
$ gcc -arch ppc -arch i386 test.c
Thes while that was running:
$ ps wwaux|grep cc1|grep -v grep
mikeash 7632 65.5 -10.3 332972 216580 p8 U+ 6:31PM 0:01.21
/usr/libexec/gcc/powerpc-apple-darwin8/4.0.1/cc1 -quiet -D__DYNAMIC__
test.c -fPIC -quiet -dumpbase test.c -auxbase test -o /var/tmp//cccaMajP.s
mikeash 7633 60.3 -10.3 333112 216872 p8 U+ 6:31PM 0:01.20
/usr/libexec/gcc/i686-apple-darwin8/4.0.1/cc1 -quiet -D__DYNAMIC__ test.c
-fPIC -quiet -dumpbase test.c -march=nocona -auxbase test -o
/var/tmp//ccjtQrfF.s