Suspected memory leak in rewrapper on Windows

[Yngve N. Pettersen]

Hello all,

Earlier this month I filed a report,
<https://issues.chromium.org/356896471>, about an apparent memory leak in
rewrapper on Windows.


As far as I can tell, there has been no action on this issue so far.


The problem is significant for me, at least, as I have to reboot my machine
at least once a week due to OOM issues (which may lead to system
instability), when I previously only had to reboot when applying Windows
Updates (once a month). The last reboot happened a few minutes ago (when I
had less than 20% memory left with Visual Studio and a Git client running
as the major memory users). The problem can be bad enough that I have had
to exit the script controlling our (incomplete) multi-week major version
Chromium update process, and resume the process later (which I consider
problematic), in order to reboot the machine.


A possibly interesting aspect is that AFAICT I do not see this on the
machines where I only use local build, remote caching, but I do see this on
my machine which is compiling remotely in at least 3-4 different checkout
environments (not in parallel, which is not possible with current reproxy
design, of course), but I frequently swap back and forth between different
projects in different checkouts.

--
Sincerely,
Yngve N. Pettersen
Vivaldi Technologies AS

[Michael Savigny]

We are unable to replicate this memory leak on our windows machines. The rewrapper process is a very short-lived one, it is also a pure go binary so it doesn't have any cgo or memory-mapping happening in it. So I would think that the large number of memory mapped files is something unrelated to rewrapper.

If you can reliably replicate the problem, our code is open-source (rewrapper entry point is here: https://github.com/bazelbuild/reclient/tree/main/cmd/rewrapper). We've got our external PR acceptance process in place, so we are happy to accept fixes for issues.

A few thoughts we've had that might be the cause:

• Because rewrapper is a go-based binary, it might 'look suspect' to certain AV or security tools. We have had at least one case where a security tool did impact rewrapper/reproxy performance due to it being a go-based binary. I'm not sure if you are able to temporarily disable anything like that and try a build.
• Given that GUI Git and Visual Studio are also chewing up large amounts of memory, it would seem like a more general issue, not rewrapper specific.
  

As a side note, if you use the autoninja tooling to execute chromium builds, you can happily run multiple concurrent builds with reproxy on the same machine (they should target different output directories, as ninja does not guarantee build correctness if you do concurrent builds against the same output directory).