[gentoo-user] www-client/chromium-63.0.3239.132

[victor romanchuk]

just noticed new use flag in recent stable chromium ebuild:

$ quse -D jumbo-build
 local:jumbo-build:www-client/chromium: Combine source files to speed up build process.

setting that significantly speeds up emerge time (tried it twice; the second attempt had the flag set)

$ qlop -gHv -d `date +%Y-%m-%d` chromium
chromium-63.0.3239.132: Fri Jan 19 03:15:43 2018: 1 hour, 47 minutes, 28 seconds
chromium-63.0.3239.132: Fri Jan 19 06:11:06 2018: 1 hour, 16 minutes, 14 seconds
chromium: 2 times

not sure if it matters in that context, but I'm using distcc and do not have ccache though

[Rich Freeman]

On Fri, Jan 19, 2018 at 12:26 AM, victor romanchuk <r...@persimplex.net> wrote:
> local:jumbo-build:www-client/chromium: Combine source files to speed up build process.
>
> setting that significantly speeds up emerge time (tried it twice; the second attempt had the flag set)
>
> $ qlop -gHv -d `date +%Y-%m-%d` chromium
> chromium-63.0.3239.132: Fri Jan 19 03:15:43 2018: 1 hour, 47 minutes, 28 seconds
> chromium-63.0.3239.132: Fri Jan 19 06:11:06 2018: 1 hour, 16 minutes, 14 seconds
> chromium: 2 times
>

There is a CPU-memory tradeoff here. Combining source files reduces
duplication of #include directives which greatly cuts down on the
number of lines of code going into the compiler, but the individual
files being compiled are larger.

I have a 12 SMT-core Ryzen 5-1600, and 16GB of RAM. I can't even
build chromium on a tmpfs because the RAM+space requirements have
grown, so I build on an SSD. Even without the tmpfs I have to reduce
make to -j11 or it will OOM during a build WITHOUT the jumbo-build
flag. So, I'm already hitting RAM limitations on build time.

That said, I've experimented with some build times and I found that I
can build chromium faster with -j8 using jumbo-build (the max # jobs I
can run reliably without OOM) than I can build it with -j11 without
using the new feature. I'll also note that to do this I have to make
sure nothing else is compiling at the same time, and sometimes I end
up stopping a container that runs mono for good measure.

I do use ccache in general with chromium but I did my benchmarking without it.

I suggest experimenting with jumbo-build, and consider reducing
parallel jobs if you run into OOM, but depending on your system you
might find it not worth the trouble.

One thing I haven't experimented with is reducing -j even further and
then moving back to a tmpfs. I could easily see a tmpfs for building
outperforming jumbo-build even if I end up at -j4 or less. Then
again, the SSD probably isn't as bad a drag as a spinning disk would
be.

I was chatting with somebody (I think on reddit) who mentioned
jumbo-build worked fine on a threadripper with 64GB of RAM (that would
be -j32 I suppose). I bet that with even a few more GB of RAM I could
probably max out my 12 SMT cores.

I can't wait to see how chromium-64 behaves. The RAM requirements
have been steadily going up. I have an older system with only 4GB RAM
and it struggles to even build chromium at all.

--
Rich

[Mick]

On my old i7 laptop it eats up all 4G of RAM and 4G of swap before it conks
out. So, I dropped the jobs to 3 and --load-average to 2, added a swapfile to
increase disk space and it now builds in around 13 hours.

I have not used jumbo-build, but perhaps I should?
--
Regards,
Mick

[Rich Freeman]

On Fri, Jan 19, 2018 at 12:04 PM, Mick <michael...@gmail.com> wrote:
>
> On my old i7 laptop it eats up all 4G of RAM and 4G of swap before it conks
> out. So, I dropped the jobs to 3 and --load-average to 2, added a swapfile to
> increase disk space and it now builds in around 13 hours.
>
> I have not used jumbo-build, but perhaps I should?

Well, you can certainly try, but since jumbo-build increases memory
use, I suspect you might not be able to get it to complete at all even
with -j1. If it did build there is a good chance it would complete
faster, even using fewer cores.

I'm not sure how exactly jumbo-build combines source files. Hopefully
it does it in a way that is consistent each time, so that ccache has
some chance of getting hits between builds. That would actually be an
advantage to compiling with many smaller units - you might get more
cache hits. That is, assuming there aren't header changes in new
releases that invalidate large number of cache entries.

--
Rich