What number to use for make -jN?
Gulshan Singh
make -j16 and make -j32." So how does that scale for my CPU? What happens if I give it a larger number than it can handle?
Jean-Baptiste Queru
you'll probably want to use -j16 (i.e. twice as many as the number of
threads that can run at the same time). If not, -j8.
Going deeper:
If you use a smaller -j than your number of cores/threads, some of
your execution units are guaranteed to stay idle. If you use exactly
the same number, some will still be idle while they wait for uncached
data to be read from storage.
In my experience, "round" numbers tend to produce better results than
"odd" numbers. e.g. on my machine (16 threads), -j24 and -j32 are
about equally fast, but intermediate values are slightly slower.
JBQ
> --
> You received this message because you are subscribed to the "Android
> Building" mailing list.
> To post to this group, send email to android-...@googlegroups.com
> To unsubscribe from this group, send email to
> android-buildi...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/android-building?hl=en
--
Jean-Baptiste M. "JBQ" Queru
Software Engineer, Android Open-Source Project, Google.
Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.
JBirdVegas
> If you use a smaller -j than your number of cores/threads, some of
> your execution units are guaranteed to stay idle. If you use exactly
> the same number, some will still be idle while they wait for uncached
> data to be read from storage.
>
>
Would we have any adverse effects from having the extra jobs waiting
for execution? ie -j12 vs -j24
Magnus Bäck
JBirdVegas <jbird...@gmail.com> wrote:
> On Feb 29, 5:17�pm, Jean-Baptiste Queru <j...@android.com> wrote:
>
> > If you use a smaller -j than your number of cores/threads, some of
> > your execution units are guaranteed to stay idle. If you use exactly
> > the same number, some will still be idle while they wait for
> > uncached data to be read from storage.
>
> Would we have any adverse effects from having the extra jobs waiting
> for execution? ie -j12 vs -j24
Possibly. It depends on e.g. how much RAM you have. When too many
processes fight over the same memory performance drops. You're not safe
just because the machine isn't swapping; memory not used for processes
waiting for a slice of the CPU could be used for caching. Secondly, too
much processes battling over the same CPUs will also cause performance
to drop.
Really, you need to measure what's best with your machine and your
workload. While there are some general truths there are also many
variables that affect the outcome.
--
Magnus B�ck
ba...@google.com
Jean-Baptiste Queru
might churn through a bit more L3 cache, have a bit more I/O
contention, but this might not have a very measurable effect.
In larger amounts, you'll start to eat into your disk cache, which
will cause more pain as file access will not hit the disk instead of
being served from/to cache.
In even larger amounts, you'll swap, and it'll be hell.
On my 24GB 16-thread machine, my default is -j32 with no problem, and
I remember testing -j48 and not seeing much difference. I use that
machine interactively during -j32 builds (nice -n19 ionice -c3) and
it's fine.
On my 12GB 16-thread machine, -j32 makes the machine painful to use. I
don't always use this machine interactively, but when I do I prefer to
keep things simple and short.
JBQ
