Fwd: docker daemon leaking memory on Managed VM

[Shantanu Tushar]

Hi,

One of our business apps is deployed on a Managed VM in AppEngine with the Go runtime. The app itself is pretty lightweight and hence we have a 0.5 CPU and 1GB RAM instance requested in app.yaml. A surprise to us, we started noticing frequent reboots of the the VM causing quite some downtime. After some analysis, we found that docker daemon was ending up consuming ~80% of RAM after 8-10 hours of uptime, making the system unresponsive (including ah health checks). This made GAE treat the host unhealthy and restarted the VM.

We tried increasing the instance RAM size to see if docker daemon is leaking or its just that it needs ~800M to work. As expected, after few days of uptime docker daemon is now consuming ~80% of 3GB of RAM (around 2.7G resident, screenshot attached).

A bit of searching online took me to discussions such as https://github.com/docker/docker/issues/12899#issuecomment-98709947 which suggests docker 1.7 contains fixes.

Has anybody faced this problem on managed VMs? If so, any possible fix/workaround?

Shantanu Tushar,

SoStronk

[Michael (Cloud Platform Support)]

Hi Shantanu!

This sort of issue would be best directed to the Public Issue Tracker for App Engine. Once there, I highly recommend creating a Defect Report and describing your issue so that our engineering team can handle it as quickly as possible and supply anyone else with this issue any further updates. Once you have created the issue, please link to it here.

Cheers!