Forced GC every 4 minutes

[Siyuan Liu]

There is this strange behavior on a Golang service we are running.

On some instances, there is a forced GC that gets triggered every 4 minutes. This can be observed when the `GODEBUG=gctrace=1` is turned on. There is a log line with `gc ... (forced)` every 4 minutes consistently on selected instances, with a few seconds off on each occurrence. It does not happen on all instances.

Based on the official golang documentation, `(forced)` is an indication that `runtime.GC()` is forcefully triggered by the application code, however, there is NO application code or third-party library code that does `runtime.GC()`. 

I am wondering if golang internally does anything that might have triggered the `runtime.GC()` - a.k.a - periodically forced GC operation?

The log line that appears every 4 minutes looks something like:

"gc 2009 @96075.949s 0%: 0.65+1314+0.13 ms clock, 39+0/19595/54342+8.2 ms cpu, 79719->80556->42707 MB, 123241 MB goal, 2 MB stacks, 1 MB globals, 60 P (forced)"

[Ian Lance Taylor]

Yes, in general a Go program will force a GC to occur if it has gone 2
minutes without doing any GC at all.

I'm not sure why you are seeing a 4 minute period, though it's
possible that I've misunderstood the code.

Ian

[S.Y. LIU]

Circle back on this. The forced GC turned out to be triggered by a module injected to the application code through the LD_PRELOAD. It was not the go library's internal 2 minutes GC that triggered it. It was hard to figure this out because the module was a total blackbox :) Thank you for trying to help.