Linker ran out of memory linking chrome.dll [Build, Win 11]

[Pavit]

Greetings,

While building chromium on Windows 22H2, 16 GB memory (15.7 of which the linker was using), AMD Ryzen 5 4000 series, the autoninja run (on a october 4 based checkout) froze at 

LINK chrome.dll [61XX2/61XX4]

It seems it won't progress because of OOME. (which is strange on the account of the whole 16 gigs being used by the linker alone)

Any guidance on how to proceed further would be appreciated.

Regards

Pavit Chhabra

[Pavit]

Thanks for the fast reply, really appreciate it.

The only other foreground process at that time was the task manager which I was using to monitor the memory/cpu consumption. (It was taking around 200 megabytes of memory and 3% cpu usage)

What was intriguing was that the cpu usage was stable at around 10% in Linking and was consistent 100% before when the objects were being built. Also is the linker using so much memory normal?

If it is normal then I might need to just wait it out? But it just seemed from the process stat that it was OOME and would not be progressing ahead.

Regards,

Pavit Chhabra

On Fri, Oct 6, 2023, 11:15 AM Sangwhan Moon <s...@chromium.org> wrote:

> On Oct 6, 2023, at 14:32, Pavit <pavitchh...@gmail.com> wrote:
>
> 

Have you tried closing some heavy processes to free up memory? Closing your browser for instance.

[K. Moon]

Linking Chrome can easily take advantage of 32 or even 64 GB of RAM. 16 GB actually might not be enough.

Linking is I/O bound, rather than CPU bound, so low CPU utilization is normal.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABNnLZsOwTDpChav-ddNvpHA11nepROGhfCEhZ66f1Vki871ZQ%40mail.gmail.com.

[Pavit]

Awesome! That makes sense.

Guess I'll try and wait for a while then.

If it's true that 16 gigs might not be enough then is there a way to provide some ssd/hdd space as VMem/Swap?

I am using -j16 in my ninja build command.

Regards

Pavit Chhabra

[K. Moon]

Adding swap should help linking complete without running out of memory, but of course it'll be slower than using RAM. It might not be that bad, depending on how busy the working set is, but you'll have to try it and find out; the results will be specific to your computing environment.

Parallelism (-j 16) has no positive effect on linking. As you've observed, a typical build will start with a lot of activity on parallel compile tasks, then end with a single long link task.

[Pavit]

Yea that makes sense! I was able to build after 5 hours of linking (I had a lot of browser windows and electron apps open so yea)


How do I make subsequent builds faster, will the autoninja automatically only build what I modified and then link all objs? Also why does this built chromium feel so slow, is this because of the debug symbols being included? 

Thanks for all the help btw! ^_^
Excited to start fixing some bugs!

[K. Moon]

Glad to hear you were able to get it working!

This page has tips on configuring Chromium builds:

https://www.chromium.org/developers/gn-build-configuration/

Autoninja always builds incrementally, but a change in a widely-used dependency may trigger building a lot more than you would expect. Also, linking must be done every time; adding RAM and using an SSD should speed this up a lot.

A smarter way to reduce linking cost is to use component builds. These are the default in debug mode.

Debug builds are less optimized, and have a lot of extra sanity checks that cost performance. (Debug symbols generally don't affect run time performance.)

Check the page about configuring the GN build for more tips. Note that whenever you change the GN configuration, you usually have to compile everything from scratch again.

[K. Moon]

One more thing: If you're building everything, you're doing a lot of extra work. Just build the "chrome" target if you only want to rebuild the browser.

[Pavit]

I've been building chrome only apparently from the gn command 😅 I just copied the commands from the build instructions page, your remark made it make sense, thanks!
I am building this on win 11 installed on a SSD but unfortunately the chromium project is on a HDD file system. So that makes sense that I/O is making linking take so much time.
I'll check out component builds and maybe try building release since I don't really need those sanity checks for doing small pointer fixes, thanks!
I'll also check out autoninja, it doesn't seem too big (especially the py script, idk about the batch file)
Thanks for all the help! ^_^

[Christian Biesinger]

If you're using `is_official_build=true` in your args.gn, that takes
an especially large amount of RAM and time.

Otherwise, you can speed things up by setting `is_component_build=true`

Christian

> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev
> ---
> You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

> To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CABNnLZuPY0kyCt9SsQLvc0dt4VmT5rVZY5VRWx_tSyvL7LJBrQ%40mail.gmail.com.

[Pavit]

I did have it the first time I built it! Then I read about component builds and made the changes, it is definitely doable now, thanks for the helpful tip!

[Sangwhan Moon]

> On Oct 6, 2023, at 14:32, Pavit <pavitchh...@gmail.com> wrote:
>
> 

[محمد علي المقداد‎]

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
    http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/F4178258-5660-473F-A9EB-708BF298B405%40chromium.org.