Mender client HTTP/2 memory leak

[Cedric Veilleux]

We distribute .mender from an nginx server.

We recently changed server and as part of the upgrade, HTTP/2 was enabled on the new server.

Using mender client 1.6, we saw updates starting to fail with Out of memory error in the mender client.

Watching mender installing an update using top, I was surprised to see the RSS consumption grow up to 500+ MB for a .mender artifact of only 240MB.

Since nothing else had changed, I tried installing the same artifact from the older server over HTTP 1.1. The RSS usage was a steady 10MB and the update succeeded.

I then suspected a buggy HTTP/2 support in the mender http client. I disabled HTTP/2 on the new nginx server and retried an update: Steady 10MB RSS usage and success.

I looked into the mender client git repos. There seems to be quite old http2 client that is vendored in. This http2 implementation is known to have memory leaks.

Reference:

https://github.com/golang/go/commit/125f52dfa8ce6df0380b0bdc66effb8afd697bda

I would suggest updating the vendored http2 client in the mender client to fix this issue.

Best regards,

Cedric

[Kristian Amlie]

Thanks, this is very useful information. We were planning to do a vendor
update soon regardless, but this puts additional pressure on that.

--
Kristian

[Kristian Amlie]

Created a task as well: https://tracker.mender.io/browse/MEN-2335

--
Kristian