Mender client HTTP/2 memory leak

22 views
Skip to first unread message

Cedric Veilleux

unread,
Jan 22, 2019, 3:04:43 PM1/22/19
to men...@lists.mender.io
We distribute .mender from an nginx server.

We recently changed server and as part of the upgrade, HTTP/2 was enabled on the new server.

Using mender client 1.6, we saw updates starting to fail with Out of memory error in the mender client.

Watching mender installing an update using top, I was surprised to see the RSS consumption grow up to 500+ MB for a .mender artifact of only 240MB.

Since nothing else had changed, I tried installing the same artifact from the older server over HTTP 1.1. The RSS usage was a steady 10MB and the update succeeded.

I then suspected a buggy HTTP/2 support in the mender http client. I disabled HTTP/2 on the new nginx server and retried an update: Steady 10MB RSS usage and success.

I looked into the mender client git repos. There seems to be quite old http2 client that is vendored in. This http2 implementation is known to have memory leaks.

Reference:

I would suggest updating the vendored http2 client in the mender client to fix this issue.

Best regards,

Cedric

Kristian Amlie

unread,
Jan 23, 2019, 2:07:30 AM1/23/19
to men...@lists.mender.io, Cedric Veilleux
Thanks, this is very useful information. We were planning to do a vendor
update soon regardless, but this puts additional pressure on that.

--
Kristian

signature.asc

Kristian Amlie

unread,
Jan 23, 2019, 5:10:58 AM1/23/19
to men...@lists.mender.io, Cedric Veilleux
Created a task as well: https://tracker.mender.io/browse/MEN-2335

--
Kristian


signature.asc
Reply all
Reply to author
Forward
0 new messages