Hi,
When I first started to create patches for thunderbird about a
dozen years ago, I read someone's blog/post that said we need 12
GB memory for comfortable build.
I had to do with 8GB of memory and it worked.
Well, C compiler has become larger, I think. Many tools including
the linker have become larger, too. Their memory requirements also
have become larger.
About half a dozen years ago, I allocated 16GB of memory to my
guest linux in VirtualBox where I have been building TB. since.
Last week, I had a hardware issue and had to replace motherboard,
and I added more memory to my PC from 32GB to 48GB.
On a hunch, I increased the memory allocated to VirtualBox from
16GB to 24 GB.
Yes, I use virtualbox to run linux on my windows PC.
Guest linux is where I create and test C-C TB patches. I know
virtualbox may not be the best environment. But I have to use MS
office tools for work, so I have to run Windows side-by-side and
there used to be quite a lot of hardware gadgets that only run
under windows. Using virtualbox was one of the solutions for this
home PC user who create patches for TB.
Before giving you the performance numbers, here is my current
hardware.
CPU: Ryzen 3700x (8C/16T), but I only allocate 7 virtual CPU to
the virtualbox.
Memory: 48 GB, but only 24GB allocated to virtualbox.
Storage: my guest linux in VirtualBox uses virtual disks both on
NVME M.2 SSD and hard disks (relatively slow at 5400 rpm).
Source tree and object tree are in virtual disks on NVME M.2 SSD.
I often change only a few C++ source files in C-C directory and
re-build TB.
So to me, relinking C-C should be fast.
ccache helps a lot.
Anyway, just relinking C-C TB takes time.
Before (with 16GB assigned to linux guest inside virtualbox):
almost 10 minutes. Slightly less. 550 seconds or so.
After (with 24 GB assigned to linux guest.): 4min 38 seconds.
x2 speed up after memory size increase (!)
The reason I think the speedup is observed is twofold:
- file buffer cache within the linux OS has become larger and thus
helps file I/O.
- less page faults. I think this counts.
I am not sure what are causing the page faults (the linker?), but
the number of page faults during C-C TB build was rather large.
I have beei observing system status using xosview visually, and I
have been puzzled by the burst of page faults observed during
build.
That is why I added 8GB to my virutalbox setup after I could
afford to add more memory to my PC. It now has 8GBx2 + 16GBx2 (ECC
memory). Before the hardware change, it had 8GB x 4 (ECC).
Well, please recall that if we need to compile sizable number of
source files,
the speed up factor is not that dramatic because it is CPU-bound
(I am yet to check it.)
But the decrease of 10 minutes to 4.5 minutes of linking by adding
8GB memory IN MY SETUP is huge for repeatedly running
edit/compile/link process on local PC.
There are some suggestions for "Requirements".
For example, "Building Firefox on Linux".
https://firefox-source-docs.mozilla.org/setup/linux_build.html
In there the "Requirements " states:
Memory: 4GB RAM minimum, 8GB+ recommended.
Disk Space: At least 30GB of free disk space.
Operating System: A 64-bit installation of Linux. It is strongly advised that you use a supported distribution; see Supported Build Hosts. We also recommend that your system is fully up-to-date.
Sure, we could build TB with 4GB RAM. I did it on 32-bit linux
until the debug symbol table became too large for 32-bit memory
space.
I switched to 64-bit linux around that time.
With virtual memory, we can build C-C TB with 4GB real memory to
be sure, but the page faults slow it down.
I think we SHOULD mention that 16GB or even 24 GB memory would be
preferable in the paragraph, not discouraging users with only 8 GB
of memory on their PCs. We should mention the reduced build time
with larger memory.
I should have increased my memory to 16GB much sooner. But my old
motherboard did not have good memory support and 32GB was the
maximum.
And I somehow had a wrong notion that 16GB was good enough. Well,
"good" is a very judgemental word.
Obviously, for "comfortable" build of C-C TB with current crop of
development tools that seem to hog memory, 24GB is definitely the
minimum IMHO now that I verified the numbers.
In that sense, I was wrong to think 16GB was good enough.
Someone might want to rewrite the paragraph in mozilla web page(s)
by adding suggested comfortable setting by investigating some
people's setups and build time observed in that setting, etc.
MS states 4GB memory is required for Windows 11.
But we should know the memory amount with which we can operate
Windows 11 and applications comfortably. Right?
The same goes for mozilla software deveelopment.
Chiaki